API Reference

This page lists all of the interfaces exposed by the treq package.

Making Requests

treq.request(method, url, **kwargs)

Make an HTTP request.

Parameters:

• method (str) – HTTP method. Example: 'GET', 'HEAD'. 'PUT', 'POST'.
• url (str) – http or https URL, which may include query arguments.
• headers (Headers or None) – Optional HTTP Headers to send with this request.
• params (dict w/ str or list/tuple of str values, list of 2-tuples, or None.) – Optional parameters to be append as the query string to the URL, any query string parameters in the URL already will be preserved.
• data (str, file-like, IBodyProducer, or None) – Optional request body.
• json (dict, list/tuple, int, string/unicode, bool, or None) – Optional JSON-serializable content to pass in body.
• reactor – Optional twisted reactor.
• persistent (bool) – Use persistent HTTP connections. Default: True
• allow_redirects (bool) – Follow HTTP redirects. Default: True
• auth (tuple of ('username', 'password').) – HTTP Basic Authentication information.
• cookies (dict or cookielib.CookieJar) – Cookies to send with this request. The HTTP kind, not the tasty kind.
• timeout (int) – Request timeout seconds. If a response is not received within this timeframe, a connection is aborted with CancelledError.
• browser_like_redirects (bool) – Use browser like redirects (i.e. Ignore RFC2616 section 10.3 and follow redirects from POST requests). Default: False
• unbuffered (bool) – Pass True to to disable response buffering. By default treq buffers the entire response body in memory.

Return type:

Deferred that fires with an IResponse provider.

treq.get(url, headers=None, **kwargs)

Make a GET request.

See treq.request()

treq.head(url, **kwargs)

Make a HEAD request.

See treq.request()

treq.post(url, data=None, **kwargs)

Make a POST request.

See treq.request()

treq.put(url, data=None, **kwargs)

Make a PUT request.

See treq.request()

treq.patch(url, data=None, **kwargs)

Make a PATCH request.

See treq.request()

treq.delete(url, **kwargs)

Make a DELETE request.

See treq.request()

Accessing Content

treq.collect(response, collector)

Incrementally collect the body of the response.

This function may only be called once for a given response.

Parameters:

• response (IResponse) – The HTTP response to collect the body from.
• collector (single argument callable) – A callable to be called each time data is available from the response body.

Return type:

Deferred that fires with None when the entire body has been read.

treq.content(response)

Read the contents of an HTTP response.

This function may be called multiple times for a response, it uses a WeakKeyDictionary to cache the contents of the response.

Parameters:response (IResponse) – The HTTP Response to get the contents of. Return type:Deferred that fires with the content as a str.

treq.text_content(response, encoding='ISO-8859-1')

Read the contents of an HTTP response and decode it with an appropriate charset, which may be guessed from the Content-Type header.

Parameters:

• response (IResponse) – The HTTP Response to get the contents of.
• encoding (str) – A charset, such as UTF-8 or ISO-8859-1, used if the response does not specify an encoding.

Return type:

Deferred that fires with a unicode string.

treq.json_content(response)

Read the contents of an HTTP response and attempt to decode it as JSON.

This function relies on content() and so may be called more than once for a given response.

Parameters:response (IResponse) – The HTTP Response to get the contents of. Return type:Deferred that fires with the decoded JSON.

HTTPClient Objects

The treq.client.HTTPClient class provides the same interface as the treq module itself.

class treq.client.HTTPClient(agent, cookiejar=None, data_to_body_producer=<InterfaceClass twisted.web.iweb.IBodyProducer>)

delete(url, **kwargs)

get(url, **kwargs)

head(url, **kwargs)

patch(url, data=None, **kwargs)

post(url, data=None, **kwargs)

put(url, data=None, **kwargs)

request(method, url, **kwargs)

Augmented Response Objects

treq.request(), treq.get(), etc. return an object which implements twisted.web.iweb.IResponse, plus a few additional convenience methods:

class treq.response._Response

collect(collector)

Incrementally collect the body of the response, per treq.collect().

Parameters:collector – A single argument callable that will be called with chunks of body data as it is received. Returns:A Deferred that fires when the entire body has been received.

content()

Read the entire body all at once, per treq.content().

Returns:A Deferred that fires with a bytes object when the entire body has been received.

json()

Collect the response body as JSON per treq.json_content().

Return type:Deferred that fires with the decoded JSON when the entire body has been read.

text(encoding='ISO-8859-1')

Read the entire body all at once as text, per treq.text_content().

Return type:A Deferred that fires with a unicode string when the entire body has been received.

history()

Get a list of all responses that (such as intermediate redirects), that ultimately ended in the current response. The responses are ordered chronologically.

Returns:A list of _Response objects

cookies()

Get a copy of this response’s cookies.

Return type:requests.cookies.RequestsCookieJar

Inherited from twisted.web.iweb.IResponse:

Variables:

• version –
• code –
• phrase –
• headers –
• length –
• request –
• previousResponse –

deliverBody(protocol)

setPreviousResponse(response)

Test Helpers

In-memory version of treq for testing.

class treq.testing.HasHeaders(headers)

Since Twisted adds headers to a request, such as the host and the content length, it’s necessary to test whether request headers CONTAIN the expected headers (the ones that are not automatically added by Twisted).

This wraps a set of headers, and can be used in an equality test against a superset if the provided headers. The headers keys are lowercased, and keys and values are compared in their bytes-encoded forms.

Headers should be provided as a mapping from strings or bytes to a list of strings or bytes.

class treq.testing.RequestSequence(sequence, async_failure_reporter)

For an example usage, see RequestSequence.consume().

Takes a sequence of:

[((method, url, params, headers, data), (code, headers, body)),
 ...]

Expects the requests to arrive in sequence order. If there are no more responses, or the request’s parameters do not match the next item’s expected request parameters, raises AssertionError.

For the expected request arguments:

• method should be bytes normalized to lowercase.
• url should be a str normalized as per the transformations in https://en.wikipedia.org/wiki/URL_normalization that (usually) preserve semantics. A URL to http://something-that-looks-like-a-directory would be normalized to http://something-that-looks-like-a-directory/ and a URL to http://something-that-looks-like-a-page/page.html remains unchanged.
• params is a dictionary mapping bytes to lists of bytes
• headers is a dictionary mapping bytes to lists of bytes - note that twisted.web.client.Agent may add its own headers though, which are not guaranteed (for instance, user-agent or content-length), so it’s better to use some kind of matcher like HasHeaders.
• data is a bytes

For the response:

• code is an integer representing the HTTP status code to return
• headers is a dictionary mapping bytes to bytes or lists of bytes
• body is a bytes

Variables:

• sequence (list) – The sequence of expected request arguments mapped to stubbed responses
• async_failure_reporter – A callable that takes a single message reporting failures—it’s asynchronous because it cannot just raise an exception—if it does, Resource.render will just convert that into a 500 response, and there will be no other failure reporting mechanism. Under Trial, this may be a twisted.logger.Logger.error, as Trial fails the test when an error is logged.

consume(**kwds)

Usage:

async_failures = []
sequence_stubs = RequestSequence([...], async_failures.append)
stub_treq = StubTreq(StringStubbingResource(sequence_stubs))
with sequence_stubs.consume(self.fail):  # self = unittest.TestCase
    stub_treq.get('http://fakeurl.com')
    stub_treq.get('http://another-fake-url.com')

self.assertEqual([], async_failures)

If there are still remaining expected requests to be made in the sequence, fails the provided test case.

Parameters:sync_failure_reporter – A callable that takes a single message reporting failures. This can just raise an exception - it does not need to be asynchronous, since the exception would not get raised within a Resource. Returns:a context manager that can be used to ensure all expected requests have been made.

consumed()

Returns:bool representing whether the entire sequence has been consumed. This is useful in tests to assert that the expected requests have all been made.

class treq.testing.RequestTraversalAgent(rootResource)

IAgent implementation that issues an in-memory request rather than going out to a real network socket.

flush()

Flush all data between pending client/server pairs.

This is only necessary if a Resource under test returns NOT_DONE_YET from its render method, making a response asynchronous. In that case, after each write from the server, pump() must be called so the client can see it.

request(method, uri, headers=None, bodyProducer=None)

Implement IAgent.request.

class treq.testing.StringStubbingResource(get_response_for)

A resource that takes a callable with 5 parameters (method, url, params, headers, data) and returns (code, headers, body).

The resource uses the callable to return a real response as a result of a request.

The parameters for the callable are:

• method, the HTTP method as bytes.
• url, the full URL of the request as text.
• params, a dictionary of query parameters mapping query keys lists of values (sorted alphabetically).
• headers, a dictionary of headers mapping header keys to a list of header values (sorted alphabetically).
• data, the request body as bytes.

The callable must return a tuple of (code, headers, body) where the code is the HTTP status code, the headers is a dictionary of bytes (unlike the headers parameter, which is a dictionary of lists), and body is a string that will be returned as the response body.

If there is a stubbing error, the return value is undefined (if an exception is raised, Resource will just eat it and return 500 in its place). The callable, or whomever creates the callable, should have a way to handle error reporting.

render(request)

Produce a response according to the stubs provided.

class treq.testing.StubTreq(resource)

A fake version of the treq module that can be used for testing that provides all the function calls exposed in treq.__all__.

Variables:resource – A Resource object that provides the fake responses

MultiPartProducer Objects

treq.multipart.MultiPartProducer is used internally when making requests which involve files.

class treq.multipart.MultiPartProducer(fields, boundary=None, cooperator=<module 'twisted.internet.task' from '/usr/lib64/python2.7/site-packages/twisted/internet/task.pyc'>)

MultiPartProducer takes parameters for a HTTP request and produces bytes in multipart/form-data format defined in RFC 2388 and RFC 2046.

The encoded request is produced incrementally and the bytes are written to a consumer.

Fields should have form: [(parameter name, value), ...]

Accepted values:

• Unicode strings (in this case parameter will be encoded with utf-8)
• Tuples with (file name, content-type, IBodyProducer objects)

Since MultiPartProducer can accept objects like IBodyProducer which cannot be read from in an event-driven manner it uses uses a Cooperator instance to schedule reads from the underlying producers. Reading is also paused and resumed based on notifications from the IConsumer provider being written to.

Variables:

• _fields – Sorted parameters, where all strings are enforced to be unicode and file objects stacked on bottom (to produce a human readable form-data request)
• _cooperate – A method like Cooperator.cooperate which is used to schedule all reads.
• boundary – The generated boundary used in form-data encoding

pauseProducing()

Temporarily suspend copying bytes from the input file to the consumer by pausing the CooperativeTask which drives that activity.

resumeProducing()

Undo the effects of a previous pauseProducing and resume copying bytes to the consumer by resuming the CooperativeTask which drives the write activity.

startProducing(consumer)

Start a cooperative task which will read bytes from the input file and write them to consumer. Return a Deferred which fires after all bytes have been written.

Parameters:consumer – Any IConsumer provider

stopProducing()

Permanently stop writing bytes from the file to the consumer by stopping the underlying CooperativeTask.