OpenStack Swift Python client binding.
OpenStack Swift client library used internally
swiftclient.client.
Connection
(authurl=None, user=None, key=None, retries=5, preauthurl=None, preauthtoken=None, snet=False, starting_backoff=1, max_backoff=64, tenant_name=None, os_options=None, auth_version='1', cacert=None, insecure=False, cert=None, cert_key=None, ssl_compression=True, retry_on_ratelimit=False, timeout=None, session=None, force_auth_retry=False)¶Bases: object
Convenience class to make requests that will also retry the request
Requests will have an X-Auth-Token header whose value is either the preauthtoken or a token obtained from the auth service using the user credentials provided as args to the constructor. If os_options includes a service_username then requests will also have an X-Service-Token header whose value is a token obtained from the auth service using the service credentials. In this case the request url will be set to the storage_url obtained from the auth service for the service user, unless this is overridden by a preauthurl.
authurl – authentication URL
user – user name to authenticate as
key – key/password to authenticate with
retries – Number of times to retry the request before failing
preauthurl – storage URL (if you have already authenticated)
preauthtoken – authentication token (if you have already authenticated) note authurl/user/key/tenant_name are not required when specifying preauthtoken
snet – use SERVICENET internal network default is False
starting_backoff – initial delay between retries (seconds)
max_backoff – maximum delay between retries (seconds)
auth_version – OpenStack auth version, default is 1.0
tenant_name – The tenant/account name, required when connecting to an auth 2.0 system.
os_options – The OpenStack options which can have tenant_id, auth_token, service_type, endpoint_type, tenant_name, object_storage_url, region_name, service_username, service_project_name, service_key
insecure – Allow to access servers without checking SSL certs. The server’s certificate will not be verified.
cert – Client certificate file to connect on SSL server requiring SSL client certificate.
cert_key – Client certificate private key file.
ssl_compression – Whether to enable compression at the SSL layer. If set to ‘False’ and the pyOpenSSL library is present an attempt to disable SSL compression will be made. This may provide a performance increase for https upload/download operations.
retry_on_ratelimit – by default, a ratelimited connection will raise an exception to the caller. Setting this parameter to True will cause a retry after a backoff.
timeout – The connect timeout for the HTTP connection.
session – A keystoneauth session object.
force_auth_retry – reset auth info even if client got unexpected error except 401 Unauthorized.
close
()¶copy_object
(container, obj, destination=None, headers=None, fresh_metadata=None, response_dict=None)¶Wrapper for copy_object()
delete_container
(container, response_dict=None, query_string=None, headers={})¶Wrapper for delete_container()
delete_object
(container, obj, query_string=None, response_dict=None, headers=None)¶Wrapper for delete_object()
get_account
(marker=None, limit=None, prefix=None, end_marker=None, full_listing=False, headers=None, delimiter=None)¶Wrapper for get_account()
get_auth
()¶get_capabilities
(url=None)¶get_container
(container, marker=None, limit=None, prefix=None, delimiter=None, end_marker=None, path=None, full_listing=False, headers=None, query_string=None)¶Wrapper for get_container()
get_object
(container, obj, resp_chunk_size=None, query_string=None, response_dict=None, headers=None)¶Wrapper for get_object()
get_service_auth
()¶head_account
(headers=None)¶Wrapper for head_account()
head_container
(container, headers=None)¶Wrapper for head_container()
head_object
(container, obj, headers=None, query_string=None)¶Wrapper for head_object()
http_connection
(url=None)¶post_account
(headers, response_dict=None, query_string=None, data=None)¶Wrapper for post_account()
post_container
(container, headers, response_dict=None)¶Wrapper for post_container()
post_object
(container, obj, headers, response_dict=None)¶Wrapper for post_object()
put_container
(container, headers=None, response_dict=None, query_string=None)¶Wrapper for put_container()
put_object
(container, obj, contents, content_length=None, etag=None, chunk_size=None, content_type=None, headers=None, query_string=None, response_dict=None)¶Wrapper for put_object()
swiftclient.client.
HTTPConnection
(url, proxy=None, cacert=None, insecure=False, cert=None, cert_key=None, ssl_compression=False, default_user_agent=None, timeout=None)¶Bases: object
Make an HTTPConnection or HTTPSConnection
url – url to connect to
proxy – proxy to connect through, if any; None by default; str of the format ‘http://127.0.0.1:8888’ to set one
cacert – A CA bundle file to use in verifying a TLS server certificate.
insecure – Allow to access servers without checking SSL certs. The server’s certificate will not be verified.
cert – Client certificate file to connect on SSL server requiring SSL client certificate.
cert_key – Client certificate private key file.
ssl_compression – SSL compression should be disabled by default and this setting is not usable as of now. The parameter is kept for backward compatibility.
default_user_agent – Set the User-Agent header on every request. If set to None (default), the user agent will be “python-swiftclient-<version>”. This may be overridden on a per-request basis by explicitly setting the user-agent header on a call to request().
timeout – socket read timeout value, passed directly to the requests library.
ClientException – Unable to handle protocol scheme
close
()¶getresponse
()¶Adapt requests response to httplib interface
putrequest
(full_path, data=None, headers=None, files=None)¶Use python-requests files upload
data – Use data generator for chunked-transfer
files – Use files for default transfer
request
(method, full_path, data=None, headers=None, files=None)¶Encode url and header, then call requests.request
swiftclient.client.
LOGGER_SENSITIVE_HEADERS
= ['x-auth-token', 'x-auth-key', 'x-service-token', 'x-storage-token', 'x-account-meta-temp-url-key', 'x-account-meta-temp-url-key-2', 'x-container-meta-temp-url-key', 'x-container-meta-temp-url-key-2', 'set-cookie']¶A list of sensitive headers to redact in logs. Note that when extending this list, the header names must be added in all lower case.
swiftclient.client.
copy_object
(url, token, container, name, destination=None, headers=None, fresh_metadata=None, http_conn=None, response_dict=None, service_token=None)¶Copy object
url – storage URL
token – auth token; if None, no token will be sent
container – container name that the source object is in
name – source object name
destination – The container and object name of the destination object in the form of /container/object; if None, the copy will use the source as the destination.
headers – additional headers to include in the request
fresh_metadata – Enables object creation that omits existing user metadata, default None
http_conn – HTTP connection object (If None, it will create the conn object)
response_dict – an optional dictionary into which to place the response - status, reason and headers
service_token – service auth token
ClientException – HTTP COPY request failed
swiftclient.client.
delete_container
(url, token, container, http_conn=None, response_dict=None, service_token=None, query_string=None, headers=None)¶Delete a container
url – storage URL
token – auth token
container – container name to delete
http_conn – a tuple of (parsed url, HTTPConnection object), (If None, it will create the conn object)
response_dict – an optional dictionary into which to place the response - status, reason and headers
service_token – service auth token
query_string – if set will be appended with ‘?’ to generated path
headers – additional headers to include in the request
ClientException – HTTP DELETE request failed
swiftclient.client.
delete_object
(url, token=None, container=None, name=None, http_conn=None, headers=None, proxy=None, query_string=None, response_dict=None, service_token=None)¶Delete object
url – storage URL
token – auth token; if None, no token will be sent
container – container name that the object is in; if None, the container name is expected to be part of the url
name – object name to delete; if None, the object name is expected to be part of the url
http_conn – a tuple of (parsed url, HTTPConnection object), (If None, it will create the conn object)
headers – additional headers to include in the request
proxy – proxy to connect through, if any; None by default; str of the format ‘http://127.0.0.1:8888’ to set one
query_string – if set will be appended with ‘?’ to generated path
response_dict – an optional dictionary into which to place the response - status, reason and headers
service_token – service auth token
ClientException – HTTP DELETE request failed
swiftclient.client.
encode_meta_headers
(headers)¶Only encode metadata headers keys
swiftclient.client.
encode_utf8
(value)¶swiftclient.client.
get_account
(url, token, marker=None, limit=None, prefix=None, end_marker=None, http_conn=None, full_listing=False, service_token=None, headers=None, delimiter=None)¶Get a listing of containers for the account.
url – storage URL
token – auth token
marker – marker query
limit – limit query
prefix – prefix query
end_marker – end_marker query
http_conn – a tuple of (parsed url, HTTPConnection object), (If None, it will create the conn object)
full_listing – if True, return a full listing, else returns a max of 10000 listings
service_token – service auth token
headers – additional headers to include in the request
delimiter – delimiter query
a tuple of (response headers, a list of containers) The response headers will be a dict and all header names will be lowercase.
ClientException – HTTP GET request failed
swiftclient.client.
get_auth
(auth_url, user, key, **kwargs)¶Get authentication/authorization credentials.
auth_version – the api version of the supplied auth params
os_options – a dict, the openstack identity service options
a tuple, (storage_url, token)
N.B. if the optional os_options parameter includes a non-empty ‘object_storage_url’ key it will override the default storage url returned by the auth service.
The snet parameter is used for Rackspace’s ServiceNet internal network implementation. In this function, it simply adds snet- to the beginning of the host name for the returned storage URL. With Rackspace Cloud Files, use of this network path causes no bandwidth charges but requires the client to be running on Rackspace’s ServiceNet network.
swiftclient.client.
get_auth_1_0
(url, user, key, snet, **kwargs)¶swiftclient.client.
get_auth_keystone
(auth_url, user, key, os_options, **kwargs)¶Authenticate against a keystone server.
We are using the keystoneclient library for authentication.
swiftclient.client.
get_capabilities
(http_conn)¶Get cluster capability infos.
http_conn – a tuple of (parsed url, HTTPConnection object)
a dict containing the cluster capabilities
ClientException – HTTP Capabilities GET failed
swiftclient.client.
get_container
(url, token, container, marker=None, limit=None, prefix=None, delimiter=None, end_marker=None, path=None, http_conn=None, full_listing=False, service_token=None, headers=None, query_string=None)¶Get a listing of objects for the container.
url – storage URL
token – auth token
container – container name to get a listing for
marker – marker query
limit – limit query
prefix – prefix query
delimiter – string to delimit the queries on
end_marker – marker query
path – path query (equivalent: “delimiter=/” and “prefix=path/”)
http_conn – a tuple of (parsed url, HTTPConnection object), (If None, it will create the conn object)
full_listing – if True, return a full listing, else returns a max of 10000 listings
service_token – service auth token
headers – additional headers to include in the request
query_string – if set will be appended with ‘?’ to generated path
a tuple of (response headers, a list of objects) The response headers will be a dict and all header names will be lowercase.
ClientException – HTTP GET request failed
swiftclient.client.
get_keystoneclient_2_0
(auth_url, user, key, os_options, **kwargs)¶swiftclient.client.
get_object
(url, token, container, name, http_conn=None, resp_chunk_size=None, query_string=None, response_dict=None, headers=None, service_token=None)¶Get an object
url – storage URL
token – auth token
container – container name that the object is in
name – object name to get
http_conn – a tuple of (parsed url, HTTPConnection object), (If None, it will create the conn object)
resp_chunk_size – if defined, chunk size of data to read. NOTE: If you specify a resp_chunk_size you must fully read the object’s contents before making another request.
query_string – if set will be appended with ‘?’ to generated path
response_dict – an optional dictionary into which to place the response - status, reason and headers
headers – an optional dictionary with additional headers to include in the request
service_token – service auth token
a tuple of (response headers, the object’s contents) The response headers will be a dict and all header names will be lowercase.
ClientException – HTTP GET request failed
swiftclient.client.
head_account
(url, token, http_conn=None, headers=None, service_token=None)¶Get account stats.
url – storage URL
token – auth token
http_conn – a tuple of (parsed url, HTTPConnection object), (If None, it will create the conn object)
headers – additional headers to include in the request
service_token – service auth token
a dict containing the response’s headers (all header names will be lowercase)
ClientException – HTTP HEAD request failed
swiftclient.client.
head_container
(url, token, container, http_conn=None, headers=None, service_token=None)¶Get container stats.
url – storage URL
token – auth token
container – container name to get stats for
http_conn – a tuple of (parsed url, HTTPConnection object), (If None, it will create the conn object)
headers – additional headers to include in the request
service_token – service auth token
a dict containing the response’s headers (all header names will be lowercase)
ClientException – HTTP HEAD request failed
swiftclient.client.
head_object
(url, token, container, name, http_conn=None, service_token=None, headers=None, query_string=None)¶Get object info
url – storage URL
token – auth token
container – container name that the object is in
name – object name to get info for
http_conn – a tuple of (parsed url, HTTPConnection object), (If None, it will create the conn object)
service_token – service auth token
headers – additional headers to include in the request
a dict containing the response’s headers (all header names will be lowercase)
ClientException – HTTP HEAD request failed
swiftclient.client.
http_connection
(*arg, **kwarg)¶tuple of (parsed url, connection object)
swiftclient.client.
http_log
(args, kwargs, resp, body)¶swiftclient.client.
logger_settings
= {'redact_sensitive_headers': True, 'reveal_sensitive_prefix': 16}¶Default behaviour is to redact header values known to contain secrets,
such as X-Auth-Key
and X-Auth-Token
. Up to the first 16 chars
may be revealed.
To disable, set the value of redact_sensitive_headers
to False
.
When header redaction is enabled, reveal_sensitive_prefix
configures the
maximum length of any sensitive header data sent to the logs. If the header
is less than twice this length, only int(len(value)/2)
chars will be
logged; if it is less than 15 chars long, even less will be logged.
swiftclient.client.
parse_header_string
(data)¶swiftclient.client.
post_account
(url, token, headers, http_conn=None, response_dict=None, service_token=None, query_string=None, data=None)¶Update an account’s metadata.
url – storage URL
token – auth token
headers – additional headers to include in the request
http_conn – a tuple of (parsed url, HTTPConnection object), (If None, it will create the conn object)
response_dict – an optional dictionary into which to place the response - status, reason and headers
service_token – service auth token
query_string – if set will be appended with ‘?’ to generated path
data – an optional message body for the request
ClientException – HTTP POST request failed
resp_headers, body
swiftclient.client.
post_container
(url, token, container, headers, http_conn=None, response_dict=None, service_token=None)¶Update a container’s metadata.
url – storage URL
token – auth token
container – container name to update
headers – additional headers to include in the request
http_conn – a tuple of (parsed url, HTTPConnection object), (If None, it will create the conn object)
response_dict – an optional dictionary into which to place the response - status, reason and headers
service_token – service auth token
ClientException – HTTP POST request failed
swiftclient.client.
post_object
(url, token, container, name, headers, http_conn=None, response_dict=None, service_token=None)¶Update object metadata
url – storage URL
token – auth token
container – container name that the object is in
name – name of the object to update
headers – additional headers to include in the request
http_conn – a tuple of (parsed url, HTTPConnection object), (If None, it will create the conn object)
response_dict – an optional dictionary into which to place the response - status, reason and headers
service_token – service auth token
ClientException – HTTP POST request failed
swiftclient.client.
prepare_unicode_headers
(self, headers)¶swiftclient.client.
put_container
(url, token, container, headers=None, http_conn=None, response_dict=None, service_token=None, query_string=None)¶Create a container
url – storage URL
token – auth token
container – container name to create
headers – additional headers to include in the request
http_conn – a tuple of (parsed url, HTTPConnection object), (If None, it will create the conn object)
response_dict – an optional dictionary into which to place the response - status, reason and headers
service_token – service auth token
query_string – if set will be appended with ‘?’ to generated path
ClientException – HTTP PUT request failed
swiftclient.client.
put_object
(url, token=None, container=None, name=None, contents=None, content_length=None, etag=None, chunk_size=None, content_type=None, headers=None, http_conn=None, proxy=None, query_string=None, response_dict=None, service_token=None)¶Put an object
url – storage URL
token – auth token; if None, no token will be sent
container – container name that the object is in; if None, the container name is expected to be part of the url
name – object name to put; if None, the object name is expected to be part of the url
contents – a string, a file-like object or an iterable to read object data from; if None, a zero-byte put will be done
content_length – value to send as content-length header; also limits the amount read from contents; if None, it will be computed via the contents or chunked transfer encoding will be used
etag – etag of contents; if None, no etag will be sent
chunk_size – chunk size of data to write; it defaults to 65536; used only if the contents object has a ‘read’ method, e.g. file-like objects, ignored otherwise
content_type – value to send as content-type header, overriding any value included in the headers param; if None and no value is found in the headers param, an empty string value will be sent
headers – additional headers to include in the request, if any
http_conn – a tuple of (parsed url, HTTPConnection object), (If None, it will create the conn object)
proxy – proxy to connect through, if any; None by default; str of the format ‘http://127.0.0.1:8888’ to set one
query_string – if set will be appended with ‘?’ to generated path
response_dict – an optional dictionary into which to place the response - status, reason and headers
service_token – service auth token
etag
ClientException – HTTP PUT request failed
swiftclient.client.
quote
(value, safe='/')¶Patched version of urllib.quote that encodes utf8 strings before quoting. On Python 3, call directly urllib.parse.quote().
swiftclient.client.
resp_header_dict
(resp)¶swiftclient.client.
safe_value
(name, value)¶Only show up to logger_settings[‘reveal_sensitive_prefix’] characters from a sensitive header.
name – Header name
value – Header value
Safe header value
swiftclient.client.
scrub_headers
(headers)¶Redact header values that can contain sensitive information that should not be logged.
headers – Either a dict or an iterable of two-element tuples
Safe dictionary of headers with sensitive information removed
swiftclient.client.
store_response
(resp, response_dict)¶store information about an operation into a dict
resp – an http response object containing the response headers
response_dict – a dict into which are placed the status, reason and a dict of lower-cased headers
swiftclient.service.
ResultsIterator
(futures)¶Bases: object
swiftclient.service.
SwiftCopyObject
(object_name, options=None)¶Bases: object
Class for specifying an object copy, allowing the destination/headers/metadata/fresh_metadata to be specified separately for each individual object. destination and fresh_metadata should be set in options
swiftclient.service.
SwiftError
(value, container=None, obj=None, segment=None, exc=None)¶Bases: Exception
swiftclient.service.
SwiftPostObject
(object_name, options=None)¶Bases: object
Class for specifying an object post, allowing the headers/metadata to be specified separately for each individual object.
swiftclient.service.
SwiftService
(options=None)¶Bases: object
Service for performing swift operations
capabilities
(url=None, refresh_cache=False)¶List the cluster capabilities.
url – Proxy URL of the cluster to retrieve capabilities.
A dictionary containing the capabilities of the cluster.
copy
(container, objects, options=None)¶Copy operations on a list of objects in a container. Destination containers will be created.
container – The container from which to copy the objects.
objects –
A list of object names (strings) or SwiftCopyObject instances containing an object name and an options dict (can be None) to override the options for that individual copy operation:
[
'object_name',
SwiftCopyObject(
'object_name',
options={
'destination': '/container/object',
'fresh_metadata': False,
...
}),
...
]
The options dict is described below.
options –
A dictionary containing options to override the global options specified during the service object creation. These options are applied to all copy operations performed by this call, unless overridden on a per object basis. The options “destination” and “fresh_metadata” do not need to be set, in this case objects will be copied onto themselves and metadata will not be refreshed. The option “destination” can also be specified in the format ‘/container’, in which case objects without an explicit destination will be copied to the destination /container/original_object_name. Combinations of multiple objects and a destination in the format ‘/container/object’ is invalid. Possible options are given below:
{
'meta': [],
'header': [],
'destination': '/container/object',
'fresh_metadata': False,
}
A generator returning the results of copying the given list of objects.
delete
(container=None, objects=None, options=None)¶Delete operations on an account, optional container and optional list of objects.
container – The container to delete or delete from.
objects – The list of objects to delete.
options –
A dictionary containing options to override the global options specified during the service object creation:
{
'yes_all': False,
'leave_segments': False,
'prefix': None,
'header': [],
}
A generator for returning the results of the delete operations. Each result yielded from the generator is either a ‘delete_container’, ‘delete_object’, ‘delete_segment’, or ‘bulk_delete’ dictionary containing the results of an individual delete operation.
download
(container=None, objects=None, options=None)¶Download operations on an account, optional container and optional list of objects.
container – The container to download from.
objects – A list of object names to download (a list of strings).
options –
A dictionary containing options to override the global options specified during the service object creation:
{
'yes_all': False,
'marker': '',
'prefix': None,
'no_download': False,
'header': [],
'skip_identical': False,
'out_directory': None,
'checksum': True,
'out_file': None,
'remove_prefix': False,
'shuffle' : False
}
A generator for returning the results of the download operations. Each result yielded from the generator is a ‘download_object’ dictionary containing the results of an individual file download.
list
(container=None, options=None)¶List operations on an account, container.
container – The container to make the list operation against.
options –
A dictionary containing options to override the global options specified during the service object creation:
{
'long': False,
'prefix': None,
'delimiter': None,
'header': []
}
A generator for returning the results of the list operation on an account or container. Each result yielded from the generator is either a ‘list_account_part’ or ‘list_container_part’, containing part of the listing.
post
(container=None, objects=None, options=None)¶Post operations on an account, container or list of objects
container – The container to make the post operation against.
objects –
A list of object names (strings) or SwiftPostObject instances containing an object name, and an options dict (can be None) to override the options for that individual post operation:
[
'object_name',
SwiftPostObject('object_name', options={...}),
...
]
The options dict is described below.
options –
A dictionary containing options to override the global options specified during the service object creation. These options are applied to all post operations performed by this call, unless overridden on a per object basis. Possible options are given below:
{
'meta': [],
'header': [],
'read_acl': None, # For containers only
'write_acl': None, # For containers only
'sync_to': None, # For containers only
'sync_key': None # For containers only
}
Either a single result dictionary in the case of a post to a container/account, or an iterator for returning the results of posts to a list of objects.
stat
(container=None, objects=None, options=None)¶Get account stats, container stats or information about a list of objects in a container.
container – The container to query.
objects – A list of object paths about which to return information (a list of strings).
options –
A dictionary containing options to override the global options specified during the service object creation. These options are applied to all stat operations performed by this call:
{
'human': False,
'header': []
}
Either a single dictionary containing stats about an account or container, or an iterator for returning the results of the stat operations on a list of objects.
upload
(container, objects, options=None)¶Upload a list of objects to a given container.
container – The container (or pseudo-folder path) to put the uploads into.
objects –
A list of file/directory names (strings) or SwiftUploadObject instances containing a source for the created object, an object name, and an options dict (can be None) to override the options for that individual upload operation:
[
'/path/to/file',
SwiftUploadObject('/path', object_name='obj1'),
...
]
The options dict is as described below.
The SwiftUploadObject source may be one of:
A file-like object (with a read method)
A string containing the path to a local file or directory
None, to indicate that we want an empty object
options –
A dictionary containing options to override the global options specified during the service object creation. These options are applied to all upload operations performed by this call, unless overridden on a per object basis. Possible options are given below:
{
'meta': [],
'header': [],
'segment_size': None,
'use_slo': False,
'segment_container': None,
'leave_segments': False,
'changed': None,
'skip_identical': False,
'fail_fast': False,
'dir_marker': False # Only for None sources
}
A generator for returning the results of the uploads.
swiftclient.service.
SwiftUploadObject
(source, object_name=None, options=None)¶Bases: object
Class for specifying an object upload, allowing the object source, name and options to be specified separately for each individual object.
swiftclient.service.
get_conn
(options)¶Return a connection building it from the options.
swiftclient.service.
get_from_queue
(q, timeout=864000)¶swiftclient.service.
get_future_result
(f, timeout=86400)¶swiftclient.service.
interruptable_as_completed
(fs, timeout=86400)¶swiftclient.service.
mkdirs
(path)¶swiftclient.service.
process_options
(options)¶swiftclient.service.
split_headers
(options, prefix='')¶Splits ‘Key: Value’ strings and returns them as a dictionary.
options – Must be one of: * an iterable of ‘Key: Value’ strings * an iterable of (‘Key’, ‘Value’) pairs * a dict of {‘Key’: ‘Value’} pairs
prefix – String to prepend to all of the keys in the dictionary. reporting.
swiftclient.exceptions.
ClientException
(msg, http_scheme='', http_host='', http_port='', http_path='', http_query='', http_status=None, http_reason='', http_device='', http_response_content='', http_response_headers=None)¶Bases: Exception
from_response
(resp, msg=None, body=None)¶swiftclient.multithreading.
ConnectionThreadPoolExecutor
(create_connection, max_workers)¶Bases: concurrent.futures.thread.ThreadPoolExecutor
A wrapper class to maintain a pool of connections alongside the thread pool. We start by creating a priority queue of connections, and each job submitted takes one of those connections (initialising if necessary) and passes it as the first arg to the executed function.
At the end of execution that connection is returned to the queue.
By using a PriorityQueue we avoid creating more connections than required. We will only create as many connections as are required concurrently.
Initializes a new ThreadPoolExecutor instance.
create_connection – callable to use to create new connections
max_workers – the maximum number of threads that can be used
submit
(fn, *args, **kwargs)¶Schedules the callable, fn, to be executed
fn – the callable to be invoked
args – the positional arguments for the callable
kwargs – the keyword arguments for the callable
a Future object representing the execution of the callable
swiftclient.multithreading.
MultiThreadingManager
(create_connection, segment_threads=10, object_dd_threads=10, object_uu_threads=10, container_threads=10)¶Bases: object
One object to manage context for multi-threading. This should make bin/swift less error-prone and allow us to test this code.
segment_threads – The number of threads allocated to segment uploads
object_dd_threads – The number of threads allocated to object download/delete jobs
object_uu_threads – The number of threads allocated to object upload/update based jobs
container_threads – The number of threads allocated to container/account level jobs
swiftclient.multithreading.
OutputManager
(print_stream=None, error_stream=None)¶Bases: object
One object to manage and provide helper functions for output.
This object is a context manager and returns itself into the context. When entering the context, two printing threads are created (see below) and they are waited on and cleaned up when exiting the context.
Also, thread-safe printing to two streams is provided. The
print_msg()
method will print to the supplied print_stream
(defaults to sys.stdout
) and the error()
method will print to the
supplied error_stream
(defaults to sys.stderr
). Both of these
printing methods will format the given string with any supplied *args
(a la printf). On Python 2, Unicode messages are encoded to utf8.
The attribute self.error_count
is incremented once per error
message printed, so an application can tell if any worker threads
encountered exceptions or otherwise called error()
on this instance.
The swift command-line tool uses this to exit non-zero if any error strings
were printed.
print_stream – The stream to which print_msg()
sends
formatted messages.
error_stream – The stream to which error()
sends formatted
messages.
On Python 2, Unicode messages are encoded to utf8.
DEFAULT_OFFSET
= 14¶error
(msg, *fmt_args)¶get_error_count
()¶print_items
(items, offset=14, skip_missing=False)¶print_msg
(msg, *fmt_args)¶print_raw
(data)¶warning
(msg, *fmt_args)¶Miscellaneous utility functions for use with Swift.
swiftclient.utils.
JSONableIterable
(iterable)¶Bases: list
swiftclient.utils.
LengthWrapper
(readable, length, md5=False)¶Bases: object
Wrap a filelike object with a maximum length.
Fix for https://github.com/kennethreitz/requests/issues/1648. It is recommended to use this class only on files opened in binary mode.
readable – The filelike object to read from.
length – The maximum amount of content that can be read from the filelike object before it is simulated to be empty.
md5 – Flag to enable calculating the MD5 of the content as it is read.
get_md5sum
()¶read
(size=-1)¶reset
¶swiftclient.utils.
ReadableToIterable
(content, chunk_size=65536, md5=False)¶Bases: object
Wrap a filelike object and act as an iterator.
It is recommended to use this class only on files opened in binary mode. Due to the Unicode changes in Python 3, files are now opened using an encoding not suitable for use with the md5 class and because of this hit the exception on every call to next. This could cause problems, especially with large files and small chunk sizes.
content – The filelike object that is yielded from.
chunk_size – The max size of each yielded item.
md5 – Flag to enable calculating the MD5 of the content as it is yielded.
get_md5sum
()¶next
()¶swiftclient.utils.
config_true_value
(value)¶Returns True if the value is either True or a string in TRUE_VALUES. Returns False otherwise. This function comes from swift.common.utils.config_true_value()
swiftclient.utils.
generate_temp_url
(path, seconds, key, method, absolute=False, prefix=False, iso8601=False, ip_range=None)¶Generates a temporary URL that gives unauthenticated access to the Swift object.
path – The full path to the Swift object or prefix if a prefix-based temporary URL should be generated. Example: /v1/AUTH_account/c/o or /v1/AUTH_account/c/prefix.
seconds – time in seconds or ISO 8601 timestamp. If absolute is False and this is the string representation of an integer, then this specifies the amount of time in seconds for which the temporary URL will be valid. If absolute is True then this specifies an absolute time at which the temporary URL will expire.
key – The secret temporary URL key set on the Swift cluster. To set a key, run ‘swift post -m “Temp-URL-Key: <substitute tempurl key here>”’
method – A HTTP method, typically either GET or PUT, to allow for this temporary URL.
absolute – if True then the seconds parameter is interpreted as a Unix timestamp, if seconds represents an integer.
prefix – if True then a prefix-based temporary URL will be generated.
iso8601 – if True, a URL containing an ISO 8601 UTC timestamp instead of a UNIX timestamp will be created.
ip_range – if a valid ip range, restricts the temporary URL to the range of ips.
ValueError – if timestamp or path is not in valid format.
the path portion of a temporary URL
swiftclient.utils.
get_body
(headers, body)¶swiftclient.utils.
iter_wrapper
(iterable)¶swiftclient.utils.
n_at_a_time
(seq, n)¶swiftclient.utils.
n_groups
(seq, n)¶swiftclient.utils.
normalize_manifest_path
(path)¶swiftclient.utils.
parse_api_response
(headers, body)¶swiftclient.utils.
prt_bytes
(num_bytes, human_flag)¶convert a number > 1024 to printable format, either in 4 char -h format as with ls -lh or return as 12 char right justified string
swiftclient.utils.
report_traceback
()¶Reports a timestamp and full traceback for a given exception.
Full traceback and timestamp.
swiftclient.utils.
split_request_headers
(options, prefix='')¶Except where otherwise noted, this document is licensed under Creative Commons Attribution 3.0 License. See all OpenStack Legal Documents.