Category:Developing Qt::Network: Difference between revisions

From Qt Wiki
Jump to navigation Jump to search
No edit summary
 
No edit summary
Line 1: Line 1:
=Introduction=
h1. Introduction


The QtNetwork module provides <span class="caps">TCP</span>/IP networking and higher layer network protocols to other modules.<br /> For example, QtWebkit and QtDeclarative.
The QtNetwork module provides TCP/IP networking and higher layer network protocols to other modules.<br />For example, QtWebkit and QtDeclarative.


The code is divided into submodules roughly corresponding to network layers.
The code is divided into submodules roughly corresponding to network layers.
Line 7: Line 7:
Platform specific code is located in _platform.cpp files, with _generic.cpp or _tcp.cpp being a basic functionality version for other platforms without a detailed port.
Platform specific code is located in _platform.cpp files, with _generic.cpp or _tcp.cpp being a basic functionality version for other platforms without a detailed port.


=Breakdown by submodule=
= Breakdown by submodule =


==kernel==
== kernel ==


The kernel contains classes which are used by multiple other layers, so don’t fit well in one place.<br /> It is a mix of “kernel” and “tools” classes, but there aren’t so many to need to further subdivide yet.
The kernel contains classes which are used by multiple other layers, so don't fit well in one place.<br />It is a mix of &quot;kernel&amp;quot; and &quot;tools&amp;quot; classes, but there aren't so many to need to further subdivide yet.


===Low level===
=== Low level ===


* qhostaddress data type for an IP address
* qhostaddress - data type for an IP address
* qnetworkinterface data type and code for enumerating interfaces (i.e. what shows up in the ifconfig/ipconfig command)
* qnetworkinterface - data type and code for enumerating interfaces (i.e. what shows up in the ifconfig/ipconfig command)


===<span class="caps">DNS</span>===
=== DNS ===


* qhostinfo used to resolve a host name to an IP address
* qhostinfo - used to resolve a host name to an IP address
* qdnslookup used for querying <span class="caps">DNS</span> records (e.g. <span class="caps">SVC</span>, MX..)
* qdnslookup - used for querying DNS records (e.g. SVC, MX..)


===Misc===
=== Misc ===


* qauthenticator data type, also contains code for calculating authentication responses. Used by http (access) and proxies (socket)
* qauthenticator - data type, also contains code for calculating authentication responses. Used by http (access) and proxies (socket)
* qnetworkproxy data type, also contains code for enumerating proxies from the operating system. (see also socket layer for proxy handlers)
* qnetworkproxy - data type, also contains code for enumerating proxies from the operating system. (see also socket layer for proxy handlers)
* qurlinfo QFileInfo for urls… used by QFtp
* qurlinfo - QFileInfo for urls… used by QFtp


==socket==
== socket ==


The socket subdirectory contains classes related to sockets. This is mainly <span class="caps">TCP</span>/IP related, but QLocalSocket is implemented on top of <span class="caps">TCP</span> only as a fallback. Note that QLocalSocket is not related to QAbstractSocket despite having a similar <span class="caps">API</span>.
The socket subdirectory contains classes related to sockets. This is mainly TCP/IP related, but QLocalSocket is implemented on top of TCP only as a fallback. Note that QLocalSocket is not related to QAbstractSocket despite having a similar API.


Sockets are implemented using socket engines. Virtual functions are called on QAbstractSocketEngine, which are implemented by QNativeSocketEngine or a proxy.
Sockets are implemented using socket engines. Virtual functions are called on QAbstractSocketEngine, which are implemented by QNativeSocketEngine or a proxy.


===local sockets===
=== local sockets ===


* qlocalsocket implements a local loopback socket
* qlocalsocket - implements a local loopback socket
* qlocalserver implements a server socket to accept incoming connections
* qlocalserver - implements a server socket to accept incoming connections
* qlocal*_unix AF_UNIX socket implementation
* qlocal*_unix - AF_UNIX socket implementation
* qlocal*_win windows named pipe implementation
* qlocal*_win - windows named pipe implementation
* qlocal*_tcp – <span class="caps">TCP</span> loopback implementation
* qlocal*''tcp - TCP loopback implementation
<br />h3. IP sockets
<br />* qabstractsocket - base class for QTcpSocket and QUdpSocket, contains most of the implementation for both<br />* qtcpsocket - TCP specific code<br />* qudpsocket - UDP specific code<br />* qtcpserver - server socket to accept incoming connections
<br />h3. socket engines
<br />* qabstractsocketengine - base class (interface)<br />* qnativesocketengine - contains cross platform code for a real socket<br />* qnativesocketengine''(platform) - contains platform specific implementation using native API
* qhttpsocketengine - implements http proxy using the CONNECT method (caching methods are dealt with in the http code in the access layer)


===IP sockets===
== SSL ==


* qabstractsocket – base class for QTcpSocket and QUdpSocket, contains most of the implementation for both
Implements the secure sockets layer.<br />See the article [[Hacking on Qts SSL Support]] for detail
* qtcpsocket – <span class="caps">TCP</span> specific code
* qudpsocket – <span class="caps">UDP</span> specific code
* qtcpserver – server socket to accept incoming connections


===socket engines===
== access ==


* qabstractsocketengine – base class (interface)
Implements QNetworkAccessManager and related classes.<br />This is the most complicated area of QtNetwork.
* qnativesocketengine – contains cross platform code for a real socket
* qnativesocketengine_(platform) – contains platform specific implementation using native <span class="caps">API</span>
* qhttpsocketengine – implements http proxy using the <span class="caps">CONNECT</span> method (caching methods are dealt with in the http code in the access layer)


==<span class="caps">SSL</span>==
QNetworkAccessManager creates a class from the QNetworkReplyImpl family, and returns it to the application as a QNetworkReply<br />There is a factory function to do this, plugins are not supported though.<br />The generic QNetworkReplyImpl has a pointer to a backend which actually does the work.<br />The non generic versions are optimised/specialised


Implements the secure sockets layer.<br /> See the article [[Hacking-on-Qts-SSL-Support|Hacking on Qts <span class="caps">SSL</span> Support]] for detail
HTTP is performed in a worker thread, to avoid disruption to the main UI thread (e.g. loading from network while scrolling)
 
==access==
 
Implements QNetworkAccessManager and related classes.<br /> This is the most complicated area of QtNetwork.
 
QNetworkAccessManager creates a class from the QNetworkReplyImpl family, and returns it to the application as a QNetworkReply<br /> There is a factory function to do this, plugins are not supported though.<br /> The generic QNetworkReplyImpl has a pointer to a backend which actually does the work.<br /> The non generic versions are optimised/specialised
 
<span class="caps">HTTP</span> is performed in a worker thread, to avoid disruption to the main UI thread (e.g. loading from network while scrolling)


Generally, a QAbstractSocket pointer is used to refer to a socket, and instantiated with either QTcpSocket or QSslSocket depending on the url scheme.
Generally, a QAbstractSocket pointer is used to refer to a socket, and instantiated with either QTcpSocket or QSslSocket depending on the url scheme.


You can also take a look at [http://peter.hartmann.tk/blog/2012/02/inside-the-qt-http-stack.html this blogpost about the Qt <span class="caps">HTTP</span> stack.] ''[peter.hartmann.tk]''
You can also take a look at &quot;this blogpost about the Qt HTTP stack.&quot;:http://peter.hartmann.tk/blog/2012/02/inside-the-qt-http-stack.html


===QNetworkAccessManager front end===
=== QNetworkAccessManager front end ===


* qnetworkaccessmanager
* qnetworkaccessmanager
Line 79: Line 70:
* qnetworkreply
* qnetworkreply


===QNetworkReply implementations===
=== QNetworkReply implementations ===


* qnetworkreplyimpl generic
* qnetworkreplyimpl - generic
* qnetworkreplydataimpl handles the data scheme
* qnetworkreplydataimpl - handles the data scheme
* qnetworkreplyfileimpl handles the file scheme
* qnetworkreplyfileimpl - handles the file scheme
* qnetworkreplyhttpimpl handles the http(s) scheme
* qnetworkreplyhttpimpl - handles the http(s) scheme
* qnetworkaccesscachebackend backend for loading from cache
* qnetworkaccesscachebackend - backend for loading from cache
* qnetworkaccessdebugpipebackend backend for unit testing
* qnetworkaccessdebugpipebackend - backend for unit testing
* qnetworkaccessfilebackend backend for file scheme (overlap/duplication?)
* qnetworkaccessfilebackend - backend for file scheme (overlap/duplication?)
* qnetworkaccessftpbackend backend for ftp scheme
* qnetworkaccessftpbackend - backend for ftp scheme


===<span class="caps">FTP</span>===
=== FTP ===


* qftp contains implementation for <span class="caps">FTP</span>
* qftp - contains implementation for FTP
* qnetworkaccessftpbackend backend for ftp scheme
* qnetworkaccessftpbackend - backend for ftp scheme


===<span class="caps">HTTP</span>===
=== HTTP ===


* qhttpmultipart container for multipart mime types when uploading
* qhttpmultipart - container for multipart mime types when uploading
* qhttpthreaddelegate worker thread task, handles the inter thread communication. Owns the QHttpNetworkConnection
* qhttpthreaddelegate - worker thread task, handles the inter thread communication. Owns the QHttpNetworkConnection
* qhttpnetworkconnection a logical connection to a server, owns multiple channels on which it sends queued requests.
* qhttpnetworkconnection - a logical connection to a server, owns multiple channels on which it sends queued requests.
* qhttpnetworkconnectionchannel a <span class="caps">TCP</span> socket connection to a server. May be reused if pipelineing is enabled.
* qhttpnetworkconnectionchannel - a TCP socket connection to a server. May be reused if pipelineing is enabled.
* qhttpnetworkheader creation and parsing of http headers. Inherited by QNetworkReply and QNetworkRequest
* qhttpnetworkheader - creation and parsing of http headers. Inherited by QNetworkReply and QNetworkRequest
* qnetworkreplyhttpimpl handles the http(s) scheme, creates the thread delegate.
* qnetworkreplyhttpimpl - handles the http(s) scheme, creates the thread delegate.


===cache and cookies===
=== cache and cookies ===


* qabstractnetworkcache base class for a url:object cache
* qabstractnetworkcache - base class for a url:object cache
* qnetworkdiskcache reference implementation for a cache backed by regular files in the filesystem
* qnetworkdiskcache - reference implementation for a cache backed by regular files in the filesystem
* qnetworkcookie data type for a cookie and associated metadata (e.g. security information)
* qnetworkcookie - data type for a cookie and associated metadata (e.g. security information)
* qnetworkcookiejar basic container for cookies, will be reimplemented by web browser or other application that needs persistant cookies
* qnetworkcookiejar - basic container for cookies, will be reimplemented by web browser or other application that needs persistant cookies


===general backend===
=== general backend ===


* qnetworkaccesscache used to hold open tcp connections that can be reused later
* qnetworkaccesscache - used to hold open tcp connections that can be reused later
* qnetworkaccessauthenticationmanager cache of authentication credentials, tried first before emitting authenticationRequired signal
* qnetworkaccessauthenticationmanager - cache of authentication credentials, tried first before emitting authenticationRequired signal


==bearer==
== bearer ==


The bearer <span class="caps">API</span> is concerned with enumerating network bearers and controlling their lifetime.<br /> It is mainly important on mobile, battery powered devices<br /> Much of the implementation is in plugins (src/plugins/bearer/(platform))<br /> There is some overlap with QNetworkInterface, but that is a passive api.<br /> The fallback bearer implementation works by polling QNetworkInterface, but the real implementations use OS specific event driven <span class="caps">API</span>s.
The bearer API is concerned with enumerating network bearers and controlling their lifetime.<br />It is mainly important on mobile, battery powered devices<br />Much of the implementation is in plugins (src/plugins/bearer/(platform))<br />There is some overlap with QNetworkInterface, but that is a passive api.<br />The fallback bearer implementation works by polling QNetworkInterface, but the real implementations use OS specific event driven APIs.


An important concept is the “service network”, which is an ordered list of network configurations<br /> (e.g. “home <span class="caps">WLAN</span>”, “office <span class="caps">WLAN</span>”, “3G data”); and the “default configuration”, which is the user’s preferred choice according to global configuration in the OS. The default configuration normally is a service network where the OS supports this.
An important concept is the &quot;service network&amp;quot;, which is an ordered list of network configurations<br />(e.g. &quot;home WLAN&amp;quot;, &quot;office WLAN&amp;quot;, &quot;3G data&amp;quot;); and the &quot;default configuration&amp;quot;, which is the user's preferred choice according to global configuration in the OS. The default configuration normally is a service network where the OS supports this.


The bearer <span class="caps">API</span> is used elsewhere in the network module, e.g. QNetworkAccessManager starts the default configuration unless it has been configured differently. QNetworkProxy::systemProxyForQuery can use a network configuration to make sure the correct proxy is used in multihoming situationss.
The bearer API is used elsewhere in the network module, e.g. QNetworkAccessManager starts the default configuration unless it has been configured differently. QNetworkProxy::systemProxyForQuery can use a network configuration to make sure the correct proxy is used in multihoming situationss.


* qnetworkconfigmanager enumeration of network configurations, notification of configuration changes
* qnetworkconfigmanager - enumeration of network configurations, notification of configuration changes
* qnetworkconfiguration data type for a network configuration
* qnetworkconfiguration - data type for a network configuration
* qnetworksession used to open/start a network configuration, and keep active for the object lifetime
* qnetworksession - used to open/start a network configuration, and keep active for the object lifetime
* qsharednetworksession A thread local singleton used by QNetworkAccessManager
* qsharednetworksession - A thread local singleton used by QNetworkAccessManager
* qbearerengine interface to the backend
* qbearerengine - interface to the backend
* qbearerplugin plugin loading
* qbearerplugin - plugin loading


=Areas for improvement=
= Areas for improvement =


* QAuthenticator (in the backend the public interface is fine)
* QAuthenticator (in the backend - the public interface is fine)
* QFtp (withdrawn as a public <span class="caps">API</span> in Qt5)
* QFtp (withdrawn as a public API in Qt5)
* QNetworkConfigurationManager (first construction blocks the thread for a noticable time)
* QNetworkConfigurationManager (first construction blocks the thread for a noticable time)
* sockets in sockets (QSslSocket, QHttpSocketEngine, QSocks5SocketEngine) each have a QAbstractSocket, leading to multiple levels of buffering and potential for racy shutdown
* sockets in sockets (QSslSocket, QHttpSocketEngine, QSocks5SocketEngine) each have a QAbstractSocket, leading to multiple levels of buffering and potential for racy shutdown
* loading ssl certificates is slow and blocking (except linux which uses openssl demand loading, which only supports files)
* loading ssl certificates is slow and blocking (except linux which uses openssl demand loading, which only supports files)
* we should support plugins for <span class="caps">URL</span> scheme handlers (e.g. ldap:) so that other Qt modules or applications themselves can get new types of url using QNetworkAccessManager.
* we should support plugins for URL scheme handlers (e.g. ldap:) so that other Qt modules or applications themselves can get new types of url using QNetworkAccessManager.
* see also [https://bugreports.qt.io/secure/IssueNavigator.jspa?mode=hide&requestId=12960 <span class="caps">JIRA</span>] ''[bugreports.qt.io]''
* see also &quot;JIRA&amp;quot;:https://bugreports.qt.io/secure/IssueNavigator.jspa?mode=hide&amp;amp;requestId=12960

Revision as of 14:43, 23 February 2015

h1. Introduction

The QtNetwork module provides TCP/IP networking and higher layer network protocols to other modules.
For example, QtWebkit and QtDeclarative.

The code is divided into submodules roughly corresponding to network layers.

Platform specific code is located in _platform.cpp files, with _generic.cpp or _tcp.cpp being a basic functionality version for other platforms without a detailed port.

Breakdown by submodule

kernel

The kernel contains classes which are used by multiple other layers, so don't fit well in one place.
It is a mix of "kernel&quot; and "tools&quot; classes, but there aren't so many to need to further subdivide yet.

Low level

  • qhostaddress - data type for an IP address
  • qnetworkinterface - data type and code for enumerating interfaces (i.e. what shows up in the ifconfig/ipconfig command)

DNS

  • qhostinfo - used to resolve a host name to an IP address
  • qdnslookup - used for querying DNS records (e.g. SVC, MX..)

Misc

  • qauthenticator - data type, also contains code for calculating authentication responses. Used by http (access) and proxies (socket)
  • qnetworkproxy - data type, also contains code for enumerating proxies from the operating system. (see also socket layer for proxy handlers)
  • qurlinfo - QFileInfo for urls… used by QFtp

socket

The socket subdirectory contains classes related to sockets. This is mainly TCP/IP related, but QLocalSocket is implemented on top of TCP only as a fallback. Note that QLocalSocket is not related to QAbstractSocket despite having a similar API.

Sockets are implemented using socket engines. Virtual functions are called on QAbstractSocketEngine, which are implemented by QNativeSocketEngine or a proxy.

local sockets

  • qlocalsocket - implements a local loopback socket
  • qlocalserver - implements a server socket to accept incoming connections
  • qlocal*_unix - AF_UNIX socket implementation
  • qlocal*_win - windows named pipe implementation
  • qlocal*tcp - TCP loopback implementation


h3. IP sockets
* qabstractsocket - base class for QTcpSocket and QUdpSocket, contains most of the implementation for both
* qtcpsocket - TCP specific code
* qudpsocket - UDP specific code
* qtcpserver - server socket to accept incoming connections
h3. socket engines
* qabstractsocketengine - base class (interface)
* qnativesocketengine - contains cross platform code for a real socket
* qnativesocketengine(platform) - contains platform specific implementation using native API

  • qhttpsocketengine - implements http proxy using the CONNECT method (caching methods are dealt with in the http code in the access layer)

SSL

Implements the secure sockets layer.
See the article Hacking on Qts SSL Support for detail

access

Implements QNetworkAccessManager and related classes.
This is the most complicated area of QtNetwork.

QNetworkAccessManager creates a class from the QNetworkReplyImpl family, and returns it to the application as a QNetworkReply
There is a factory function to do this, plugins are not supported though.
The generic QNetworkReplyImpl has a pointer to a backend which actually does the work.
The non generic versions are optimised/specialised

HTTP is performed in a worker thread, to avoid disruption to the main UI thread (e.g. loading from network while scrolling)

Generally, a QAbstractSocket pointer is used to refer to a socket, and instantiated with either QTcpSocket or QSslSocket depending on the url scheme.

You can also take a look at "this blogpost about the Qt HTTP stack.":http://peter.hartmann.tk/blog/2012/02/inside-the-qt-http-stack.html

QNetworkAccessManager front end

  • qnetworkaccessmanager
  • qnetworkrequest
  • qnetworkreply

QNetworkReply implementations

  • qnetworkreplyimpl - generic
  • qnetworkreplydataimpl - handles the data scheme
  • qnetworkreplyfileimpl - handles the file scheme
  • qnetworkreplyhttpimpl - handles the http(s) scheme
  • qnetworkaccesscachebackend - backend for loading from cache
  • qnetworkaccessdebugpipebackend - backend for unit testing
  • qnetworkaccessfilebackend - backend for file scheme (overlap/duplication?)
  • qnetworkaccessftpbackend - backend for ftp scheme

FTP

  • qftp - contains implementation for FTP
  • qnetworkaccessftpbackend - backend for ftp scheme

HTTP

  • qhttpmultipart - container for multipart mime types when uploading
  • qhttpthreaddelegate - worker thread task, handles the inter thread communication. Owns the QHttpNetworkConnection
  • qhttpnetworkconnection - a logical connection to a server, owns multiple channels on which it sends queued requests.
  • qhttpnetworkconnectionchannel - a TCP socket connection to a server. May be reused if pipelineing is enabled.
  • qhttpnetworkheader - creation and parsing of http headers. Inherited by QNetworkReply and QNetworkRequest
  • qnetworkreplyhttpimpl - handles the http(s) scheme, creates the thread delegate.

cache and cookies

  • qabstractnetworkcache - base class for a url:object cache
  • qnetworkdiskcache - reference implementation for a cache backed by regular files in the filesystem
  • qnetworkcookie - data type for a cookie and associated metadata (e.g. security information)
  • qnetworkcookiejar - basic container for cookies, will be reimplemented by web browser or other application that needs persistant cookies

general backend

  • qnetworkaccesscache - used to hold open tcp connections that can be reused later
  • qnetworkaccessauthenticationmanager - cache of authentication credentials, tried first before emitting authenticationRequired signal

bearer

The bearer API is concerned with enumerating network bearers and controlling their lifetime.
It is mainly important on mobile, battery powered devices
Much of the implementation is in plugins (src/plugins/bearer/(platform))
There is some overlap with QNetworkInterface, but that is a passive api.
The fallback bearer implementation works by polling QNetworkInterface, but the real implementations use OS specific event driven APIs.

An important concept is the "service network&quot;, which is an ordered list of network configurations
(e.g. "home WLAN&quot;, "office WLAN&quot;, "3G data&quot;); and the "default configuration&quot;, which is the user's preferred choice according to global configuration in the OS. The default configuration normally is a service network where the OS supports this.

The bearer API is used elsewhere in the network module, e.g. QNetworkAccessManager starts the default configuration unless it has been configured differently. QNetworkProxy::systemProxyForQuery can use a network configuration to make sure the correct proxy is used in multihoming situationss.

  • qnetworkconfigmanager - enumeration of network configurations, notification of configuration changes
  • qnetworkconfiguration - data type for a network configuration
  • qnetworksession - used to open/start a network configuration, and keep active for the object lifetime
  • qsharednetworksession - A thread local singleton used by QNetworkAccessManager
  • qbearerengine - interface to the backend
  • qbearerplugin - plugin loading

Areas for improvement

  • QAuthenticator (in the backend - the public interface is fine)
  • QFtp (withdrawn as a public API in Qt5)
  • QNetworkConfigurationManager (first construction blocks the thread for a noticable time)
  • sockets in sockets (QSslSocket, QHttpSocketEngine, QSocks5SocketEngine) each have a QAbstractSocket, leading to multiple levels of buffering and potential for racy shutdown
  • loading ssl certificates is slow and blocking (except linux which uses openssl demand loading, which only supports files)
  • we should support plugins for URL scheme handlers (e.g. ldap:) so that other Qt modules or applications themselves can get new types of url using QNetworkAccessManager.
  • see also "JIRA&quot;:https://bugreports.qt.io/secure/IssueNavigator.jspa?mode=hide&amp;requestId=12960