gecko-dev/netwerk/protocol/http
Patrick McManus fcec57856d bug 603512 - large objects block pipelines r=honzab
the type and state patch tries hard not to form pipelines behind resources that
could become head of line blockers. But of course that requires the ability to
predict the future, and won't be perfect.

This patch reacts to a transaction that has a large response body (defined by
either a large content-length header or actually reading a large number of
chunked bytes) by cancelling any transactions that have been pipelined down the
same connection and rescheduling them elsewhere. It also changes the type of
the connection to "solo", which prevents new transactions from being pipelined
onto this one and provides class-specific negative feedback to the pipeline
manager so that near-future requests to the same host of the same type (e.g.
general) will not be pipelined but other types (e.g. img or js/css) can still
do that.

Content-Length is ideal, because it allows us to identify the problem so early.
But even actually reading the document for a fairly long time gives it a fairly
high probability of not ending soon. (i.e. long document sizes are spread over
a larger range than small ones. duh.)

The pref network.http.pipelining.maxsize controls the threshold. I set the
default at 300KB, which is roughly the bandwidth delay product of a 2mbit 120ms
rtt connection and 1 rtt is mostly what you are giving up by canceling it on
one connection and sending it on another. (modulo maybe needing a handshake).
2012-03-22 19:39:31 -04:00
..
HttpBaseChannel.cpp Bug 598304 - XHR rewrites non-POST methods upon 301/302 redirects. r=bz 2012-02-10 14:12:51 +01:00
HttpBaseChannel.h bug 528288 - reland spdy after libxul weightloss a=khuey CLOSED TREE 2011-12-13 10:55:50 -05:00
HttpChannelChild.cpp Backed out changeset 5aaec7f808b6 2012-03-15 20:55:44 -07:00
HttpChannelChild.h Backed out changeset 5aaec7f808b6 2012-03-15 20:55:44 -07:00
HttpChannelParent.cpp Backed out changeset 5aaec7f808b6 2012-03-15 20:55:44 -07:00
HttpChannelParent.h Bug 646512 HttpChannelParent::OnStartRequest deep-copies nsHttpHeaderArray. r=jduell 2012-01-19 14:26:43 -08:00
HttpChannelParentListener.cpp Bug 675553 - Switch from PRBool to bool on a CLOSED TREE , r=bsmedberg,khuey,bz,cjones 2011-09-28 23:19:26 -07:00
HttpChannelParentListener.h Bug 591707 - e10s: handle redirects from HTTP to a different protocol, r=jduell, a=blocking-fennec2.0b3+ 2010-11-23 23:56:06 +01:00
Makefile.in Backed out changeset f65247c7647a (bug 729133) 2012-02-26 13:50:34 +13:00
PHttpChannel.ipdl Bug 646512 HttpChannelParent::OnStartRequest deep-copies nsHttpHeaderArray. r=jduell 2012-01-19 14:26:43 -08:00
PHttpChannelParams.h Bug 675553 - Switch from PRBool to bool on a CLOSED TREE , r=bsmedberg,khuey,bz,cjones 2011-09-28 23:19:26 -07:00
README
SpdySession.cpp bug 603512 - large objects block pipelines r=honzab 2012-03-22 19:39:31 -04:00
SpdySession.h bug 603512 - large objects block pipelines r=honzab 2012-03-22 19:39:31 -04:00
SpdyStream.cpp bug 727943 spdy broke caldav (and probly webdav) methods r=honzab 2012-02-19 14:29:09 -05:00
SpdyStream.h bug 708415 spdy make stream frame writes to session atomic r=honzab 2012-01-26 00:15:26 -05:00
TimingStruct.h Bug 658822 - Fix implementation of DNS timing in nsITimedChannel; r=jduell 2011-05-22 21:29:42 -07:00
ipdl.mk
nsAHttpConnection.h bug 603512 - large objects block pipelines r=honzab 2012-03-22 19:39:31 -04:00
nsAHttpTransaction.h bug 599164 pipeline with type and state r=honzab 2012-03-22 19:39:31 -04:00
nsHttp.cpp Bug 729940 - Part 2: Stop using crappy hash functions in Gecko. r=bz 2012-03-12 18:53:18 -04:00
nsHttp.h bug 599164 pipeline with type and state r=honzab 2012-03-22 19:39:31 -04:00
nsHttpActivityDistributor.cpp Bug 675553 - Switch from PRBool to bool on a CLOSED TREE , r=bsmedberg,khuey,bz,cjones 2011-09-28 23:19:26 -07:00
nsHttpActivityDistributor.h Rollup of bug 645263 and bug 646259: Switch to mozilla:: sync primitives. r=cjones,dbaron,doublec,ehsan src=bsmedberg 2011-03-31 23:29:02 -05:00
nsHttpAtomList.h bug 597684 Implement HTTP Assoc-req and Banned Pipelines on nsHttpConnectionInfo r=honzab 2012-03-22 19:39:31 -04:00
nsHttpAuthCache.cpp Bug 675553 - Switch from PRBool to bool on a CLOSED TREE , r=bsmedberg,khuey,bz,cjones 2011-09-28 23:19:26 -07:00
nsHttpAuthCache.h Bug 675553 - Switch from PRBool to bool on a CLOSED TREE , r=bsmedberg,khuey,bz,cjones 2011-09-28 23:19:26 -07:00
nsHttpAuthManager.cpp
nsHttpAuthManager.h
nsHttpBasicAuth.cpp Bug 690892 - Replace PR_TRUE/PR_FALSE with true/false on mozilla-central; rs=dbaron 2011-10-17 10:59:28 -04:00
nsHttpBasicAuth.h
nsHttpChannel.cpp bug 599164 pipeline with type and state r=honzab 2012-03-22 19:39:31 -04:00
nsHttpChannel.h bug 597684 Implement HTTP Assoc-req and Banned Pipelines on nsHttpConnectionInfo r=honzab 2012-03-22 19:39:31 -04:00
nsHttpChannelAuthProvider.cpp Back out be556c99ef81 and 13c3c54d067b (bug 627616) for orange 2012-02-12 10:53:31 -08:00
nsHttpChannelAuthProvider.h Bug 675553 - Switch from PRBool to bool on a CLOSED TREE , r=bsmedberg,khuey,bz,cjones 2011-09-28 23:19:26 -07:00
nsHttpChunkedDecoder.cpp Bug 690892 - Replace PR_TRUE/PR_FALSE with true/false on mozilla-central; rs=dbaron 2011-10-17 10:59:28 -04:00
nsHttpChunkedDecoder.h Bug 690892 - Replace PR_TRUE/PR_FALSE with true/false on mozilla-central; rs=dbaron 2011-10-17 10:59:28 -04:00
nsHttpConnection.cpp bug 665885 respect keepalive: "max=" attribute r=honzab 2012-03-22 19:39:31 -04:00
nsHttpConnection.h bug 603512 - large objects block pipelines r=honzab 2012-03-22 19:39:31 -04:00
nsHttpConnectionInfo.cpp bug 599164 pipeline with type and state r=honzab 2012-03-22 19:39:31 -04:00
nsHttpConnectionInfo.h bug 599164 pipeline with type and state r=honzab 2012-03-22 19:39:31 -04:00
nsHttpConnectionMgr.cpp bug 603512 - large objects block pipelines r=honzab 2012-03-22 19:39:31 -04:00
nsHttpConnectionMgr.h bug 603512 - large objects block pipelines r=honzab 2012-03-22 19:39:31 -04:00
nsHttpDigestAuth.cpp Bug 690892 - Replace PR_TRUE/PR_FALSE with true/false on mozilla-central; rs=dbaron 2011-10-17 10:59:28 -04:00
nsHttpDigestAuth.h Bug 675553 - Switch from PRBool to bool on a CLOSED TREE , r=bsmedberg,khuey,bz,cjones 2011-09-28 23:19:26 -07:00
nsHttpHandler.cpp bug 603512 - large objects block pipelines r=honzab 2012-03-22 19:39:31 -04:00
nsHttpHandler.h bug 603512 - large objects block pipelines r=honzab 2012-03-22 19:39:31 -04:00
nsHttpHeaderArray.cpp bug 704227 muddle through on some minor problems with content-length r=jduell 2012-02-09 14:47:30 -05:00
nsHttpHeaderArray.h bug 704227 muddle through on some minor problems with content-length r=jduell 2012-02-09 14:47:30 -05:00
nsHttpNTLMAuth.cpp Bug 452781 - Allow specifying wildcard that matches all simple netbiosnames in network.automatic-ntlm-auth.trusted-uris, r=honzab 2012-02-17 16:24:31 +01:00
nsHttpNTLMAuth.h Bug 573043 - Enable Extended Protection (channel and service bindng) for NTLM authentication, r=jmathies 2011-11-09 18:18:59 +01:00
nsHttpPipeline.cpp bug 603512 - large objects block pipelines r=honzab 2012-03-22 19:39:31 -04:00
nsHttpPipeline.h bug 599164 pipeline with type and state r=honzab 2012-03-22 19:39:31 -04:00
nsHttpRequestHead.cpp Bug 675553 - Switch from PRBool to bool on a CLOSED TREE , r=bsmedberg,khuey,bz,cjones 2011-09-28 23:19:26 -07:00
nsHttpRequestHead.h Bug 675553 - Switch from PRBool to bool on a CLOSED TREE , r=bsmedberg,khuey,bz,cjones 2011-09-28 23:19:26 -07:00
nsHttpResponseHead.cpp Bug 714302: add suport for HTTP status code 308 r=mcmanus 2012-03-21 16:13:42 -04:00
nsHttpResponseHead.h Bug 690892 - Replace PR_TRUE/PR_FALSE with true/false on mozilla-central; rs=dbaron 2011-10-17 10:59:28 -04:00
nsHttpTransaction.cpp bug 603512 - large objects block pipelines r=honzab 2012-03-22 19:39:31 -04:00
nsHttpTransaction.h bug 603512 - large objects block pipelines r=honzab 2012-03-22 19:39:31 -04:00
nsIHttpActivityObserver.idl
nsIHttpAuthManager.idl
nsIHttpAuthenticableChannel.idl
nsIHttpAuthenticator.idl Bug 616809 Remove CRs from netwerk r=biesi a=jst DONTBUILD 2011-02-06 00:04:14 +00:00
nsIHttpChannel.idl
nsIHttpChannelAuthProvider.idl
nsIHttpChannelChild.idl Bug 646512 HttpChannelParent::OnStartRequest deep-copies nsHttpHeaderArray. r=jduell 2012-01-19 14:26:43 -08:00
nsIHttpChannelInternal.idl bug 528288 - reland spdy after libxul weightloss a=khuey CLOSED TREE 2011-12-13 10:55:50 -05:00
nsIHttpEventSink.idl
nsIHttpHeaderVisitor.idl Bug 668642 - Make nsIHttpHeaderVisitor a [function] interface. r=bz 2011-06-30 23:55:56 +02:00
nsIHttpProtocolHandler.idl Bug 646428 - Remove nsIHttpProtocolHandler's language attribute. r=bz 2011-04-06 17:17:28 +02:00

README

                                                        Darin Fisher
                                                        darin@netscape.com
                                                        8/8/2001

                            HTTP DESIGN NOTES


CLASS BREAKDOWN

  nsHttpHandler
    - implements nsIProtocolHandler
    - manages preferences
    - owns the authentication cache
    - holds references to frequently used services

  nsHttpChannel
    - implements nsIHttpChannel
    - talks to the cache
    - initiates http transactions
    - processes http response codes
    - intercepts progress notifications
  
  nsHttpConnection
    - implements nsIStreamListener & nsIStreamProvider
    - talks to the socket transport service
    - feeds data to its transaction object
    - routes progress notifications

  nsHttpConnectionInfo
    - identifies a connection

  nsHttpTransaction
    - implements nsIRequest
    - encapsulates a http request and response
    - parses incoming data

  nsHttpChunkedDecoder
    - owned by a transaction
    - removes chunked decoding
  
  nsHttpRequestHead
    - owns a nsHttpHeaderArray
    - knows how to fill a request buffer

  nsHttpResponseHead
    - owns a nsHttpHeaderArray
    - knows how to parse response lines
    - performs common header manipulations/calculations

  nsHttpHeaderArray
    - stores http "<header>:<value>" pairs

  nsHttpAuthCache
    - stores authentication credentials for http auth domains

  nsHttpBasicAuth
    - implements nsIHttpAuthenticator
    - generates BASIC auth credentials from user:pass


ATOMS

  nsHttp:: (header namespace)

  eg. nsHttp::Content_Length


TRANSACTION MODEL

  InitiateTransaction -> ActivateConnection -> AsyncWrite, AsyncRead

  The channel creates transactions, and passes them to the handler via
  InitiateTransaction along with a nsHttpConnectionInfo object 
  identifying the requested connection.  The handler either dispatches
  the transaction immediately or queues it up to be dispatched later,
  depending on whether or not the limit on the number of connections
  to the requested server has been reached.  Once the transaction can
  be run, the handler looks for an idle connection or creates a new
  connection, and then (re)activates the connection, assigning it the
  new transaction.

  Once activated the connection ensures that it has a socket transport,
  and then calls AsyncWrite and AsyncRead on the socket transport.  This
  begins the process of talking to the server.  To minimize buffering,
  socket transport thread-proxying is completely disabled (using the flags
  DONT_PROXY_LISTENER | DONT_PROXY_PROVIDER | DONT_PROXY_OBSERVER with
  both AsyncWrite and AsyncRead).  This means that the nsHttpConnection's
  OnStartRequest, OnDataAvailable, OnDataWritable, and OnStopRequest
  methods will execute on the socket transport thread.

  The transaction defines (non-virtual) OnDataReadable, OnDataWritable, and
  OnStopTransaction methods, which the connection calls in response to
  its OnDataAvailable, OnDataWritable, and OnStopRequest methods, respectively.
  The transaction owns a nsStreamListenerProxy created by the channel, which
  it uses to transfer data from the socket thread over to the client's thread.
  To mimize buffering, the transaction implements nsIInputStream, and passes
  itself to the stream listener proxy's OnDataAvailable.  In this way, we
  have effectively wedged the response parsing between the socket and the
  thread proxy's buffer.  When read, the transaction turns around and reads
  from the socket using the buffer passed to it.  The transaction scans the
  buffer for headers, removes them as they are detected, and copies the headers
  into its nsHttpResponseHead object.  The rest of the data remains in the
  buffer, and is proxied over to the client's thread to be handled first by the
  http channel and eventually by the client.

  There are several other major design factors, including:

    - transaction cancelation
    - progress notification
    - SSL tunneling
    - chunked decoding
    - thread safety
    - premature EOF detection and transaction restarting
    - pipelining (not yet implemented)


CACHING

<EOF>