Misc

Exceptions

exception swift.common.exceptions.AuditException

Bases: exceptions.Exception

exception swift.common.exceptions.AuthException

Bases: exceptions.Exception

exception swift.common.exceptions.ChunkReadTimeout(seconds=None, exception=None)

Bases: eventlet.timeout.Timeout

exception swift.common.exceptions.ChunkWriteTimeout(seconds=None, exception=None)

Bases: eventlet.timeout.Timeout

exception swift.common.exceptions.ConnectionTimeout(seconds=None, exception=None)

Bases: eventlet.timeout.Timeout

exception swift.common.exceptions.DriveNotMounted

Bases: exceptions.Exception

exception swift.common.exceptions.LockTimeout(seconds=None, msg=None)

Bases: swift.common.exceptions.MessageTimeout

exception swift.common.exceptions.MessageTimeout(seconds=None, msg=None)

Bases: eventlet.timeout.Timeout

Constraints

swift.common.constraints.ACCOUNT_LISTING_LIMIT

Max container list length of a get request for an account

swift.common.constraints.CONTAINER_LISTING_LIMIT

Max object list length of a get request for a container

swift.common.constraints.MAX_FILE_SIZE

Max file size allowed for objects

swift.common.constraints.MAX_META_COUNT

Max number of metadata items

swift.common.constraints.MAX_META_NAME_LENGTH

Max length of the name of a key for metadata

swift.common.constraints.MAX_META_OVERALL_SIZE

Max overall size of metadata

swift.common.constraints.MAX_META_VALUE_LENGTH

Max length of the value of a key for metadata

swift.common.constraints.MAX_OBJECT_NAME_LENGTH

Max object name length

swift.common.constraints.check_float(string)

Helper function for checking if a string can be converted to a float.

Parameters:string – string to be verified as a float
Returns:True if the string can be converted to a float, False otherwise
swift.common.constraints.check_metadata(req, target_type)

Check metadata sent in the request headers.

Parameters:
  • req – request object
  • target_type – str: one of: object, container, or account: indicates which type the target storage for the metadata is
Raises HTTPBadRequest:
 

bad metadata

swift.common.constraints.check_mount(root, drive)

Verify that the path to the device is a mount point and mounted. This allows us to fast fail on drives that have been unmounted because of issues, and also prevents us for accidently filling up the root partition.

Parameters:
  • root – base path where the devices are mounted
  • drive – drive name to be checked
Returns:

True if it is a valid mounted device, False otherwise

swift.common.constraints.check_object_creation(req, object_name)

Check to ensure that everything is alright about an object to be created.

Parameters:
  • req – HTTP request object
  • object_name – name of object to be created
Raises:
  • HTTPRequestEntityTooLarge – the object is too large
  • HTTPLengthRequered – missing content-length header and not a chunked request
  • HTTPBadRequest – missing or bad content-type header, or bad metadata
swift.common.constraints.check_utf8(string)

Validate if a string is valid UTF-8.

Parameters:string – string to be validated
Returns:True if the string is valid utf-8, False otherwise

Utils

Miscellaneous utility functions for use with Swift.

class swift.common.utils.NamedLogger(logger, server)

Bases: object

Cheesy version of the LoggerAdapter available in Python 3

class swift.common.utils.NullLogger

A no-op logger for eventlet wsgi.

swift.common.utils.audit_location_generator(devices, datadir, mount_check=True, logger=None)

Given a devices path and a data directory, yield (path, device, partition) for all files in that directory

Parameters:
  • devices – parent directory of the devices to be audited
  • datadir – a directory located under self.devices. This should be one of the DATADIR constants defined in the account, container, and object servers.
  • mount_check – Flag to check if a mount check should be performed on devices
  • logger – a logger object
swift.common.utils.cache_from_env(env)

Get memcache connection pool from the environment (which had been previously set by the memcache middleware

Parameters:env – wsgi environment dict
Returns:swift.common.memcached.MemcacheRing from environment
swift.common.utils.compute_eta(start_time, current_value, final_value)

Compute an ETA. Now only if we could also have a progress bar...

Parameters:
  • start_time – Unix timestamp when the operation began
  • current_value – Current value
  • final_value – Final value
Returns:

ETA as a tuple of (length of time, unit of time) where unit of time is one of (‘h’, ‘m’, ‘s’)

swift.common.utils.drop_buffer_cache(fd, offset, length)

Drop ‘buffer’ cache for the given range of the given file.

Parameters:
  • fd – file descriptor
  • offset – start offset
  • length – length
swift.common.utils.drop_privileges(user)

Sets the userid of the current process

Parameters:user – User id to change privileges to
swift.common.utils.fallocate(fd, size)

Pre-allocate disk space for a file file.

Parameters:
  • fd – file descriptor
  • size – size to allocate (in bytes)
swift.common.utils.get_logger(conf, name=None)

Get the current system logger using config settings.

Log config and defaults:

log_facility = LOG_LOCAL0
log_level = INFO
log_name = swift
Parameters:
  • conf – Configuration dict to read settings from
  • name – Name of the logger
swift.common.utils.get_param(req, name, default=None)

Get parameters from an HTTP request ensuring proper handling UTF-8 encoding.

Parameters:
  • req – Webob request object
  • name – parameter name
  • default – result to return if the parameter is not found
Returns:

HTTP request parameter value

swift.common.utils.get_time_units(time_amount)

Get a nomralized length of time in the largest unit of time (hours, minutes, or seconds.)

Parameters:time_amount – length of time in seconds
Returns:A touple of (length of time, unit of time) where unit of time is one of (‘h’, ‘m’, ‘s’)
swift.common.utils.hash_path(account, container=None, object=None, raw_digest=False)

Get the connonical hash for an account/container/object

Parameters:
  • account – Account
  • container – Container
  • object – Object
  • raw_digest – If True, return the raw version rather than a hex digest
Returns:

hash string

swift.common.utils.item_from_env(env, item_name)

Get a value from the wsgi environment

Parameters:
  • env – wsgi environment dict
  • item_name – name of item to get
Returns:

the value from the environment

swift.common.utils.iter_devices_partitions(devices_dir, item_type)

Iterate over partitions accross all devices.

Parameters:
  • devices_dir – Path to devices
  • item_type – One of ‘accounts’, ‘containers’, or ‘objects’
Returns:

Each iteration returns a tuple of (device, partition)

swift.common.utils.load_libc_function(func_name)

Attempt to find the function in libc, otherwise return a no-op func.

Parameters:func_name – name of the function to pull from libc.
swift.common.utils.lock_parent_directory(filename, timeout=10)

Context manager that acquires a lock on the parent directory of the given file path. This will block until the lock can be acquired, or the timeout time has expired (whichever occurs first).

Parameters:
  • filename – file path of the parent directory to be locked
  • timeout – timeout (in seconds)
swift.common.utils.lock_path(*args, **kwds)

Context manager that acquires a lock on a directory. This will block until the lock can be acquired, or the timeout time has expired (whichever occurs first).

Parameters:
  • directory – directory to be locked
  • timeout – timeout (in seconds)
swift.common.utils.mkdirs(path)

Ensures the path is a directory or makes it if not. Errors if the path exists but is a file or on permissions failure.

Parameters:path – path to create
swift.common.utils.normalize_timestamp(timestamp)

Format a timestamp (string or numeric) into a standardized xxxxxxxxxx.xxxxx format.

Parameters:timestamp – unix timestamp
Returns:normalized timestamp as a string
swift.common.utils.readconf(conf, section_name=None, log_name=None, defaults=None)

Read config file and return config items as a dict

Parameters:
  • conf – path to config file
  • section_name – config section to read (will return all sections if not defined)
  • log_name – name to be used with logging (will use section_name if not defined)
  • defaults – dict of default values to pre-populate the config with
Returns:

dict of config items

swift.common.utils.renamer(old, new)

Attempt to fix^H^H^Hhide race conditions like empty object directories being removed by backend processes during uploads, by retrying.

Parameters:
  • old – old path to be renamed
  • new – new path to be renamed to
swift.common.utils.split_path(path, minsegs=1, maxsegs=None, rest_with_last=False)

Validate and split the given HTTP request path.

Examples:

['a'] = split_path('/a')
['a', None] = split_path('/a', 1, 2)
['a', 'c'] = split_path('/a/c', 1, 2)
['a', 'c', 'o/r'] = split_path('/a/c/o/r', 1, 3, True)
Parameters:
  • path – HTTP Request path to be split
  • minsegs – Minimum number of segments to be extracted
  • maxsegs – Maximum number of segments to be extracted
  • rest_with_last – If True, trailing data will be returned as part of last segment. If False, and there is trailing data, raises ValueError.
Returns:

list of segments with a length of maxsegs (non-existant segments will return as None)

swift.common.utils.storage_directory(datadir, partition, hash)

Get the storage directory

Parameters:
  • datadir – Base data directory
  • partition – Partition
  • hash – Account, container or object hash
Returns:

Storage directory

Remove any file in a given path that that was last modified before mtime.

Parameters:path – path to remove file from
Mtime :timestamp of oldest file to keep
swift.common.utils.whataremyips()

Get the machine’s ip addresses using ifconfig

Returns:list of Strings of IPv4 ip addresses
swift.common.utils.write_pickle(obj, dest, tmp)

Ensure that a pickle file gets written to disk. The file is first written to a tmp location, ensure it is synced to disk, then perform a move to its final location

Parameters:
  • obj – python object to be pickled
  • dest – path of final destination file
  • tmp – path to tmp to use

Auth

class swift.common.middleware.auth.DevAuth(app, conf)

Bases: object

Auth Middleware that uses the dev auth server.

authorize(req)

Returns None if the request is authorized to continue or a standard WSGI response callable if not.

denied_response(req)

Returns a standard WSGI response callable with the status of 403 or 401 depending on whether the REMOTE_USER is set or not.

get_groups(token, memcache_client=None)

Get groups for the given token.

If memcache_client is set, token credentials will be cached appropriately.

With a cache miss, or no memcache_client, the configurated external authentication server will be queried for the group information.

Parameters:
  • token – Token to validate and return a group string for.
  • memcache_client – Memcached client to use for caching token credentials; None if no caching is desired.
Returns:

None if the token is invalid or a string containing a comma separated list of groups the authenticated user is a member of. The first group in the list is also considered a unique identifier for that user.

swift.common.middleware.auth.filter_factory(global_conf, **local_conf)

Returns a WSGI filter app for use with paste.deploy.

ACLs

swift.common.middleware.acl.clean_acl(name, value)

Returns a cleaned ACL header value, validating that it meets the formatting requirements for standard Swift ACL strings.

The ACL format is:

[item[,item...]]

Each item can be a group name to give access to or a referrer designation to grant or deny based on the HTTP Referer header.

The referrer designation format is:

.r:[-]value

The .r can also be .ref, .referer, or .referrer; though it will be shortened to just .r for decreased character count usage.

The value can be * to specify any referrer host is allowed access, a specific host name like www.example.com, or if it has a leading period . or leading *. it is a domain name specification, like .example.com or *.example.com. The leading minus sign - indicates referrer hosts that should be denied access.

Referrer access is applied in the order they are specified. For example, .r:.example.com,.r:-thief.example.com would allow all hosts ending with .example.com except for the specific host thief.example.com.

Example valid ACLs:

.r:*
.r:*,.r:-.thief.com
.r:*,.r:.example.com,.r:-thief.example.com
.r:*,.r:-.thief.com,bobs_account,sues_account:sue
bobs_account,sues_account:sue

Example invalid ACLs:

.r:
.r:-

Also, .r designations aren’t allowed in headers whose names include the word ‘write’.

ACLs that are “messy” will be cleaned up. Examples:

Original Cleaned
bob, sue bob,sue
bob , sue bob,sue
bob,,,sue bob,sue
.referrer : * .r:*
.ref:*.example.com .r:.example.com
Parameters:
  • name – The name of the header being cleaned, such as X-Container-Read or X-Container-Write.
  • value – The value of the header being cleaned.
Returns:

The value, cleaned of extraneous formatting.

Raises ValueError:
 

If the value does not meet the ACL formatting requirements; the error message will indicate why.

swift.common.middleware.acl.parse_acl(acl_string)

Parses a standard Swift ACL string into a referrers list and groups list.

See clean_acl() for documentation of the standard Swift ACL format.

Parameters:acl_string – The standard Swift ACL string to parse.
Returns:A tuple of (referrers, groups) where referrers is a list of referrer designations (without the leading .r:) and groups is a list of groups to allow access.
swift.common.middleware.acl.referrer_allowed(referrer, referrer_acl)

Returns True if the referrer should be allowed based on the referrer_acl list (as returned by parse_acl()).

See clean_acl() for documentation of the standard Swift ACL format.

Parameters:
  • referrer – The value of the HTTP Referer header.
  • referrer_acl – The list of referrer designations as returned by parse_acl().
Returns:

True if the referrer should be allowed; False if not.

WSGI

Client

Cloud Files client library used internally

exception swift.common.client.ClientException(msg, http_scheme='', http_host='', http_port='', http_path='', http_query='', http_status=0, http_reason='', http_device='')

Bases: exceptions.Exception

class swift.common.client.Connection(authurl, user, key, retries=5, preauthurl=None, preauthtoken=None, snet=False)

Bases: object

Convenience class to make requests that will also retry the request

delete_container(container)

Wrapper for delete_container()

delete_object(container, obj)

Wrapper for delete_object()

get_account(marker=None, limit=None, prefix=None, full_listing=False)

Wrapper for get_account()

get_auth()
get_container(container, marker=None, limit=None, prefix=None, delimiter=None, full_listing=False)

Wrapper for get_container()

get_object(container, obj, resp_chunk_size=None)

Wrapper for get_object()

head_account()

Wrapper for head_account()

head_container(container)

Wrapper for head_container()

head_object(container, obj)

Wrapper for head_object()

http_connection()
post_account(headers)

Wrapper for post_account()

post_container(container, headers)

Wrapper for post_container()

post_object(container, obj, headers)

Wrapper for post_object()

put_container(container, headers=None)

Wrapper for put_container()

put_object(container, obj, contents, content_length=None, etag=None, chunk_size=65536, content_type=None, headers=None)

Wrapper for put_object()

swift.common.client.delete_container(url, token, container, http_conn=None)

Delete a container

Parameters:
  • url – storage URL
  • token – auth token
  • container – container name to delete
  • http_conn – HTTP connection object (If None, it will create the conn object)
Raises ClientException:
 

HTTP DELETE request failed

swift.common.client.delete_object(url, token, container, name, http_conn=None)

Delete object

Parameters:
  • url – storage URL
  • token – auth token
  • container – container name that the object is in
  • name – object name to delete
  • http_conn – HTTP connection object (If None, it will create the conn object)
Raises ClientException:
 

HTTP DELETE request failed

swift.common.client.get_account(url, token, marker=None, limit=None, prefix=None, http_conn=None, full_listing=False)

Get a listing of containers for the account.

Parameters:
  • url – storage URL
  • token – auth token
  • marker – marker query
  • limit – limit query
  • prefix – prefix query
  • http_conn – HTTP connection object (If None, it will create the conn object)
  • full_listing – if True, return a full listing, else returns a max of 10000 listings
Returns:

a tuple of (response headers, a list of containers) The response headers will be a dict and all header names will be lowercase.

Raises ClientException:
 

HTTP GET request failed

swift.common.client.get_auth(url, user, key, snet=False)

Get authentication/authorization credentials.

The snet parameter is used for Rackspace’s ServiceNet internal network implementation. In this function, it simply adds snet- to the beginning of the host name for the returned storage URL. With Rackspace Cloud Files, use of this network path causes no bandwidth charges but requires the client to be running on Rackspace’s ServiceNet network.

Parameters:
  • url – authentication/authorization URL
  • user – user to authenticate as
  • key – key or password for authorization
  • snet – use SERVICENET internal network (see above), default is False
Returns:

tuple of (storage URL, auth token)

Raises ClientException:
 

HTTP GET request to auth URL failed

swift.common.client.get_container(url, token, container, marker=None, limit=None, prefix=None, delimiter=None, http_conn=None, full_listing=False)

Get a listing of objects for the container.

Parameters:
  • url – storage URL
  • token – auth token
  • container – container name to get a listing for
  • marker – marker query
  • limit – limit query
  • prefix – prefix query
  • delimeter – string to delimit the queries on
  • http_conn – HTTP connection object (If None, it will create the conn object)
  • full_listing – if True, return a full listing, else returns a max of 10000 listings
Returns:

a tuple of (response headers, a list of objects) The response headers will be a dict and all header names will be lowercase.

Raises ClientException:
 

HTTP GET request failed

swift.common.client.get_object(url, token, container, name, http_conn=None, resp_chunk_size=None)

Get an object

Parameters:
  • url – storage URL
  • token – auth token
  • container – container name that the object is in
  • name – object name to get
  • http_conn – HTTP connection object (If None, it will create the conn object)
  • resp_chunk_size – if defined, chunk size of data to read. NOTE: If you specify a resp_chunk_size you must fully read the object’s contents before making another request.
Returns:

a tuple of (response headers, the object’s contents) The response headers will be a dict and all header names will be lowercase.

Raises ClientException:
 

HTTP GET request failed

swift.common.client.head_account(url, token, http_conn=None)

Get account stats.

Parameters:
  • url – storage URL
  • token – auth token
  • http_conn – HTTP connection object (If None, it will create the conn object)
Returns:

a dict containing the response’s headers (all header names will be lowercase)

Raises ClientException:
 

HTTP HEAD request failed

swift.common.client.head_container(url, token, container, http_conn=None)

Get container stats.

Parameters:
  • url – storage URL
  • token – auth token
  • container – container name to get stats for
  • http_conn – HTTP connection object (If None, it will create the conn object)
Returns:

a dict containing the response’s headers (all header names will be lowercase)

Raises ClientException:
 

HTTP HEAD request failed

swift.common.client.head_object(url, token, container, name, http_conn=None)

Get object info

Parameters:
  • url – storage URL
  • token – auth token
  • container – container name that the object is in
  • name – object name to get info for
  • http_conn – HTTP connection object (If None, it will create the conn object)
Returns:

a dict containing the response’s headers (all header names will be lowercase)

Raises ClientException:
 

HTTP HEAD request failed

swift.common.client.http_connection(url)

Make an HTTPConnection or HTTPSConnection

Parameters:url – url to connect to
Returns:tuple of (parsed url, connection object)
Raises ClientException:
 Unable to handle protocol scheme
swift.common.client.post_account(url, token, headers, http_conn=None)

Update an account’s metadata.

Parameters:
  • url – storage URL
  • token – auth token
  • headers – additional headers to include in the request
  • http_conn – HTTP connection object (If None, it will create the conn object)
Raises ClientException:
 

HTTP POST request failed

swift.common.client.post_container(url, token, container, headers, http_conn=None)

Update a container’s metadata.

Parameters:
  • url – storage URL
  • token – auth token
  • container – container name to update
  • headers – additional headers to include in the request
  • http_conn – HTTP connection object (If None, it will create the conn object)
Raises ClientException:
 

HTTP POST request failed

swift.common.client.post_object(url, token, container, name, headers, http_conn=None)

Update object metadata

Parameters:
  • url – storage URL
  • token – auth token
  • container – container name that the object is in
  • name – name of the object to update
  • headers – additional headers to include in the request
  • http_conn – HTTP connection object (If None, it will create the conn object)
Raises ClientException:
 

HTTP POST request failed

swift.common.client.put_container(url, token, container, headers=None, http_conn=None)

Create a container

Parameters:
  • url – storage URL
  • token – auth token
  • container – container name to create
  • headers – additional headers to include in the request
  • http_conn – HTTP connection object (If None, it will create the conn object)
Raises ClientException:
 

HTTP PUT request failed

swift.common.client.put_object(url, token, container, name, contents, content_length=None, etag=None, chunk_size=65536, content_type=None, headers=None, http_conn=None)

Put an object

Parameters:
  • url – storage URL
  • token – auth token
  • container – container name that the object is in
  • name – object name to put
  • contents – a string or a file like object to read object data from
  • content_length – value to send as content-length header
  • etag – etag of contents
  • chunk_size – chunk size of data to write
  • content_type – value to send as content-type header
  • headers – additional headers to include in the request
  • http_conn – HTTP connection object (If None, it will create the conn object)
Returns:

etag from server response

Raises ClientException:
 

HTTP PUT request failed

swift.common.client.quote(value, safe='/')

Patched version of urllib.quote that encodes utf8 strings before quoting

Direct Client

Internal client library for making calls directly to the servers rather than through the proxy.

swift.common.direct_client.direct_delete_container(node, part, account, container, conn_timeout=5, response_timeout=15, headers={})
swift.common.direct_client.direct_delete_object(node, part, account, container, obj, conn_timeout=5, response_timeout=15, headers={})

Delete object directly from the object server.

Parameters:
  • node – node dictionary from the ring
  • part – partition the container is on
  • account – account name
  • container – container name
  • obj – object name
  • conn_timeout – timeout in seconds for establishing the connection
  • response_timeout – timeout in seconds for getting the response
Returns:

response from server

swift.common.direct_client.direct_get_container(node, part, account, container, marker=None, limit=None, prefix=None, delimiter=None, conn_timeout=5, response_timeout=15)

Get container listings directly from the container server.

Parameters:
  • node – node dictionary from the ring
  • part – partition the container is on
  • account – account name
  • container – container name
  • marker – marker query
  • limit – query limit
  • prefix – prefix query
  • delimeter – delimeter for the query
  • conn_timeout – timeout in seconds for establishing the connection
  • response_timeout – timeout in seconds for getting the response
Returns:

a tuple of (response headers, a list of objects) The response headers will be a dict and all header names will be lowercase.

swift.common.direct_client.direct_get_object(node, part, account, container, obj, conn_timeout=5, response_timeout=15, resp_chunk_size=None)

Get object directly from the object server.

Parameters:
  • node – node dictionary from the ring
  • part – partition the container is on
  • account – account name
  • container – container name
  • obj – object name
  • conn_timeout – timeout in seconds for establishing the connection
  • response_timeout – timeout in seconds for getting the response
  • resp_chunk_size – if defined, chunk size of data to read.
Returns:

a tuple of (response headers, the object’s contents) The response headers will be a dict and all header names will be lowercase.

swift.common.direct_client.direct_head_container(node, part, account, container, conn_timeout=5, response_timeout=15)

Request container information directly from the container server.

Parameters:
  • node – node dictionary from the ring
  • part – partition the container is on
  • account – account name
  • container – container name
  • conn_timeout – timeout in seconds for establishing the connection
  • response_timeout – timeout in seconds for getting the response
Returns:

a dict containing the response’s headers (all header names will be lowercase)

swift.common.direct_client.direct_head_object(node, part, account, container, obj, conn_timeout=5, response_timeout=15)

Request object information directly from the object server.

Parameters:
  • node – node dictionary from the ring
  • part – partition the container is on
  • account – account name
  • container – container name
  • obj – object name
  • conn_timeout – timeout in seconds for establishing the connection
  • response_timeout – timeout in seconds for getting the response
Returns:

a dict containing the response’s headers (all header names will be lowercase)

swift.common.direct_client.direct_put_object(node, part, account, container, name, contents, content_length=None, etag=None, content_type=None, headers=None, conn_timeout=5, response_timeout=15, resp_chunk_size=None)

Put object directly from the object server.

Parameters:
  • node – node dictionary from the ring
  • part – partition the container is on
  • account – account name
  • container – container name
  • name – object name
  • contents – a string to read object data from
  • content_length – value to send as content-length header
  • etag – etag of contents
  • content_type – value to send as content-type header
  • headers – additional headers to include in the request
  • conn_timeout – timeout in seconds for establishing the connection
  • response_timeout – timeout in seconds for getting the response
  • chunk_size – if defined, chunk size of data to send.
Returns:

etag from the server response

swift.common.direct_client.quote(value, safe='/')
swift.common.direct_client.retry(func, *args, **kwargs)

Helper function to retry a given function a number of times.

Parameters:
  • func – callable to be called
  • retries – number of retries
  • error_log – logger for errors
  • args – arguments to send to func
  • kwargs – keyward arguments to send to func (if retries or error_log are sent, they will be deleted from kwargs before sending on to func)
Returns:

restult of func

Buffered HTTP

Monkey Patch httplib.HTTPResponse to buffer reads of headers. This can improve performance when making large numbers of small HTTP requests. This module also provides helper functions to make HTTP connections using BufferedHTTPResponse.

Warning

If you use this, be sure that the libraries you are using do not access the socket directly (xmlrpclib, I’m looking at you :/), and instead make all calls through httplib.

class swift.common.bufferedhttp.BufferedHTTPConnection(host, port=None, strict=None, timeout=<object object at 0x14643d0>, source_address=None)

Bases: httplib.HTTPConnection

HTTPConnection class that uses BufferedHTTPResponse

response_class

alias of BufferedHTTPResponse

class swift.common.bufferedhttp.BufferedHTTPResponse(sock, debuglevel=0, strict=0, method=None)

Bases: httplib.HTTPResponse

HTTPResponse class that buffers reading of headers

swift.common.bufferedhttp.http_connect(ipaddr, port, device, partition, method, path, headers=None, query_string=None, ssl=False)

Helper function to create an HTTPConnection object. If ssl is set True, HTTPSConnection will be used. However, if ssl=False, BufferedHTTPConnection will be used, which is buffered for backend Swift services.

Parameters:
  • ipaddr – IPv4 address to connect to
  • port – port to connect to
  • device – device of the node to query
  • partition – partition on the device
  • method – HTTP method to request (‘GET’, ‘PUT’, ‘POST’, etc.)
  • path – request path
  • headers – dictionary of headers
  • query_string – request query string
  • ssl – set True if SSL should be used (default: False)
Returns:

HTTPConnection object

swift.common.bufferedhttp.http_connect_raw(ipaddr, port, method, path, headers=None, query_string=None, ssl=False)

Helper function to create an HTTPConnection object. If ssl is set True, HTTPSConnection will be used. However, if ssl=False, BufferedHTTPConnection will be used, which is buffered for backend Swift services.

Parameters:
  • ipaddr – IPv4 address to connect to
  • port – port to connect to
  • method – HTTP method to request (‘GET’, ‘PUT’, ‘POST’, etc.)
  • path – request path
  • headers – dictionary of headers
  • query_string – request query string
  • ssl – set True if SSL should be used (default: False)
Returns:

HTTPConnection object

Healthcheck

class swift.common.middleware.healthcheck.HealthCheckMiddleware(app, *args, **kwargs)

Bases: object

Healthcheck middleware used for monitoring.

If the path is /healthcheck, it will respond with “OK” in the body

GET(req)

Returns a 200 response with “OK” in the body.

MemCacheD

Lucid comes with memcached: v1.4.2. Protocol documentation for that version is at:

http://github.com/memcached/memcached/blob/1.4.2/doc/protocol.txt

class swift.common.memcached.MemcacheRing(servers, connect_timeout=0.3, io_timeout=2.0, tries=3)

Bases: object

Simple, consistent-hashed memcache client.

decr(key, delta=1, timeout=0)

Decrements a key which has a numeric value by delta. If the key can’t be found, it’s added as 0. Memcached will treat data values below 0 as 0 with incr/decr.

Parameters:
  • key – key
  • delta – amount to subtract to the value of key (or set as the value if the key is not found)
  • timeout – ttl in memcache
delete(key)

Deletes a key/value pair from memcache.

Parameters:key – key to be deleted
get(key)

Gets the object specified by key. It will also unpickle the object before returning if it is pickled in memcache.

Parameters:key – key
Returns:value of the key in memcache
get_multi(keys, server_key)

Gets multiple values from memcache for the given keys.

Parameters:
  • keys – keys for values to be retrieved from memcache
  • servery_key – key to use in determining which server in the ring is used
Returns:

list of values

incr(key, delta=1, timeout=0)

Increments a key which has a numeric value by delta. If the key can’t be found, it’s added as delta.

Parameters:
  • key – key
  • delta – amount to add to the value of key (or set as the value if the key is not found)
  • timeout – ttl in memcache
set(key, value, serialize=True, timeout=0)

Set a key/value pair in memcache

Parameters:
  • key – key
  • value – value
  • serialize – if True, value is pickled before sending to memcache
  • timeout – ttl in memcache
set_multi(mapping, server_key, serialize=True, timeout=0)

Sets multiple key/value pairs in memcache.

Parameters:
  • mapping – dictonary of keys and values to be set in memcache
  • servery_key – key to use in determining which server in the ring is used
  • serialize – if True, value is pickled before sending to memcache
  • timeout – ttl for memcache

Ratelimit

class swift.common.middleware.ratelimit.RateLimitMiddleware(app, conf, logger=None)

Bases: object

Rate limiting middleware

Rate limits requests on both an Account and Container level. Limits are configurable.

get_container_maxrate(container_size)

Returns number of requests allowed per second for given container size.

get_ratelimitable_key_tuples(req_method, account_name, container_name=None, obj_name=None)

Returns a list of key (used in memcache), ratelimit tuples. Keys should be checked in order.

Parameters:
  • req_method – HTTP method
  • account_name – account name from path
  • container_name – container name from path
  • obj_name – object name from path
handle_ratelimit(req, account_name, container_name, obj_name)

Performs rate limiting and account white/black listing. Sleeps if necessary.

Parameters:
  • account_name – account name from path
  • container_name – container name from path
  • obj_name – object name from path
swift.common.middleware.ratelimit.filter_factory(global_conf, **local_conf)

paste.deploy app factory for creating WSGI proxy apps.

Table Of Contents

Previous topic

Developer’s Authorization

This Page