System Utilities¶
Helper functions for working with the underlying system. These are mostly os dependent, only working on linux, osx, and bsd. In almost all cases they’re best-effort, providing None if the lookup fails.
Changed in version 1.3.0: Dropped the get_* prefix from several function names. The old names still work, but are deprecated aliases.
Changed in version 1.5.0: Added the SYSTEM_CALL_TIME global, which tracks total time spent making system commands.
Module Overview:
is_windows - checks if we're running on windows
is_mac - checks if we're running on a mac
is_gentoo - checks if we're running on gentoo
is_slackware - checks if we're running on slackware
is_bsd - checks if we're running on the bsd family of operating systems
is_available - determines if a command is available on this system
is_running - determines if a given process is running
size_of - provides the memory usage of an object
call - runs the given system command and provides back the results
name_by_pid - gets the name for a process by the given pid
pid_by_name - gets the pid for a process by the given name
pid_by_port - gets the pid for a process listening to a given port
pid_by_open_file - gets the pid for the process with an open file
pids_by_user - provides processes owned by a user
cwd - provides the current working directory for a given process
user - provides the user a process is running under
start_time - provides the unix timestamp when the process started
tail - provides lines from the end of a file
bsd_jail_id - provides the BSD jail id a given process is running within
bsd_jail_path - provides the path of the given BSD jail
is_tarfile - checks if the given path is a tarball
expand_path - expands relative paths and ~ entries
files_with_suffix - provides files with the given suffix
get_process_name - provides our process' name
set_process_name - changes our process' name
-
stem.util.system.
Status
(enum)¶ State of a subprocess.
New in version 1.6.0.
Status
Description
PENDING
not yet started
RUNNING
currently being performed
DONE
completed successfully
FAILED
failed with an exception
-
stem.util.system.
argc_t
¶ alias of
stem.util.system.LP_c_char_p
-
exception
stem.util.system.
CallError
(msg, command, exit_status, runtime, stdout, stderr)[source]¶ Bases:
OSError
Error response when making a system call. This is an OSError subclass with additional information about the process. Depending on the nature of the error not all of these attributes will be available.
- Variables
msg (str) – exception string
command (str) – command that was ran
exit_status (int) – exit code of the process
runtime (float) – time the command took to run
stdout (str) – stdout of the process
stderr (str) – stderr of the process
-
exception
stem.util.system.
CallTimeoutError
(msg, command, exit_status, runtime, stdout, stderr, timeout)[source]¶ Bases:
stem.util.system.CallError
Error response when making a system call that has timed out.
New in version 1.6.0.
- Variables
timeout (float) – time we waited
-
class
stem.util.system.
DaemonTask
(runner, args=None, priority=15, start=False)[source]¶ Bases:
object
Invokes the given function in a subprocess, returning the value.
New in version 1.6.0.
- Variables
runner (function) – function to be invoked by the subprocess
args (tuple) – arguments to provide to the subprocess
priority (int) – subprocess nice priority
status (stem.util.system.State) – state of the subprocess
runtime (float) – seconds subprocess took to complete
result (object) – return value of subprocess if successful
error (exception) – exception raised by subprocess if it failed
-
stem.util.system.
is_windows
()[source]¶ Checks if we are running on Windows.
- Returns
bool to indicate if we’re on Windows
-
stem.util.system.
is_mac
()[source]¶ Checks if we are running on Mac OSX.
- Returns
bool to indicate if we’re on a Mac
-
stem.util.system.
is_gentoo
()[source]¶ Checks if we’re running on Gentoo.
- Returns
bool to indicate if we’re on Gentoo
-
stem.util.system.
is_slackware
()[source]¶ Checks if we are running on a Slackware system.
- Returns
bool to indicate if we’re on a Slackware system
-
stem.util.system.
is_bsd
()[source]¶ Checks if we are within the BSD family of operating systems. This currently recognizes Macs, FreeBSD, and OpenBSD but may be expanded later.
- Returns
bool to indicate if we’re on a BSD OS
-
stem.util.system.
is_available
(command, cached=True)[source]¶ Checks the current PATH to see if a command is available or not. If more than one command is present (for instance “ls -a | grep foo”) then this just checks the first.
Note that shell (like cd and ulimit) aren’t in the PATH so this lookup will try to assume that it’s available. This only happends for recognized shell commands (those in SHELL_COMMANDS).
- Parameters
command (str) – command to search for
cached (bool) – makes use of available cached results if True
- Returns
True if an executable we can use by that name exists in the PATH, False otherwise
-
stem.util.system.
is_running
(command)[source]¶ Checks for if a process with a given name or pid is running.
Changed in version 1.6.0: Added support for list and pid arguments.
- Parameters
command (str,list,int) – process name if a str, multiple process names if a list, or pid if an int to be checked
- Returns
True if the process is running, False if it’s not among ps results, and None if ps can’t be queried
-
stem.util.system.
size_of
(obj, exclude=None)[source]¶ Provides the approximate memory usage of an object. This can recurse tuples, lists, deques, dicts, and sets. To teach this function to inspect additional object types expand SIZE_RECURSES…
stem.util.system.SIZE_RECURSES[SomeClass] = SomeClass.get_elements
New in version 1.6.0.
- Parameters
obj (object) – object to provide the size of
exclude (set) – object ids to exclude from size estimation
- Returns
int with the size of the object in bytes
- Raises
NotImplementedError if using PyPy
-
stem.util.system.
name_by_pid
(pid)[source]¶ Attempts to determine the name a given process is running under (not including arguments). This uses…
1. Information from /proc 2. ps -p <pid> -o command
- Parameters
pid (int) – process id of the process to be queried
- Returns
str with the process name, None if it can’t be determined
-
stem.util.system.
pid_by_name
(process_name, multiple=False)[source]¶ Attempts to determine the process id for a running process, using…
1. pgrep -x <name> 2. pidof <name> 3. ps -o pid -C <name> (linux) ps axc | egrep " <name>$" (bsd) 4. lsof -tc <name> 5. tasklist | str <name>.exe
- Parameters
process_name (str) – process name for which to fetch the pid
multiple (bool) – provides a list of all pids if True, otherwise results with multiple processes are discarded
- Returns
Response depends upon the ‘multiple’ argument as follows…
if False then this provides an int with the process id or None if it can’t be determined
if True then this provides a list of all int process ids, and an empty list if it can’t be determined
-
stem.util.system.
pid_by_port
(port)[source]¶ Attempts to determine the process id for a process with the given port, using…
1. netstat -npltu | grep 127.0.0.1:<port> 2. sockstat -4l -P tcp -p <port> 3. lsof -wnP -iTCP -sTCP:LISTEN | grep ":<port>"
Most queries limit results to listening TCP connections. This function likely won’t work on Mac OSX.
- Parameters
port (int) – port where the process we’re looking for is listening
- Returns
int with the process id, None if it can’t be determined
-
stem.util.system.
pid_by_open_file
(path)[source]¶ Attempts to determine the process id for a process with the given open file, using…
lsof -w <path>
- Parameters
path (str) – location of the socket file to query against
- Returns
int with the process id, None if it can’t be determined
-
stem.util.system.
pids_by_user
(user)[source]¶ Provides processes owned by a given user.
New in version 1.5.0.
- Parameters
user (str) – user to look up processes for
- Returns
list with the process ids, None if it can’t be determined
-
stem.util.system.
cwd
(pid)[source]¶ Provides the working directory of the given process.
- Parameters
pid (int) – process id of the process to be queried
- Returns
str with the absolute path for the process’ present working directory, None if it can’t be determined
-
stem.util.system.
user
(pid)[source]¶ Provides the user a process is running under.
- Parameters
pid (int) – process id of the process to be queried
- Returns
str with the username a process is running under, None if it can’t be determined
-
stem.util.system.
start_time
(pid)[source]¶ Provides the unix timestamp when the given process started.
- Parameters
pid (int) – process id of the process to be queried
- Returns
float for the unix timestamp when the process began, None if it can’t be determined
-
stem.util.system.
tail
(target, lines=None)[source]¶ Provides lines of a file starting with the end. For instance, ‘tail -n 50 /tmp/my_log’ could be done with…
reversed(list(tail('/tmp/my_log', 50)))
- Parameters
target (str,file) – path or file object to read from
lines (int) – number of lines to read
- Returns
generator that reads lines, starting with the end
- Raises
IOError if unable to read the file
-
stem.util.system.
bsd_jail_id
(pid)[source]¶ Gets the jail id for a process. These seem to only exist for FreeBSD (this style for jails does not exist on Linux, OSX, or OpenBSD).
- Parameters
pid (int) – process id of the jail id to be queried
- Returns
int for the jail id, zero if this can’t be determined
-
stem.util.system.
bsd_jail_path
(jid)[source]¶ Provides the path of the given FreeBSD jail.
- Parameters
jid (int) – jail id to be queried
- Returns
str of the path prefix, None if this can’t be determined
-
stem.util.system.
is_tarfile
(path)[source]¶ Returns if the path belongs to a tarfile or not.
New in version 1.2.0.
- Parameters
path (str) – path to be checked
- Returns
True if the path belongs to a tarball, False otherwise
-
stem.util.system.
expand_path
(path, cwd=None)[source]¶ Provides an absolute path, expanding tildes with the user’s home and appending a current working directory if the path was relative.
- Parameters
path (str) – path to be expanded
cwd (str) – current working directory to expand relative paths with, our process’ if this is None
- Returns
str of the path expanded to be an absolute path, never with an ending slash
-
stem.util.system.
files_with_suffix
(base_path, suffix)[source]¶ Iterates over files in a given directory, providing filenames with a certain suffix.
New in version 1.2.0.
- Parameters
base_path (str) – directory to be iterated over
suffix (str) – filename suffix to look for
- Returns
iterator that yields the absolute path for files with the given suffix
-
stem.util.system.
call
(command, default = UNDEFINED, ignore_exit_status = False)[source]¶ Issues a command in a subprocess, blocking until completion and returning the results. This is not actually ran in a shell so pipes and other shell syntax are not permitted.
Changed in version 1.5.0: Providing additional information upon failure by raising a CallError. This is a subclass of OSError, providing backward compatibility.
Changed in version 1.5.0: Added env argument.
Changed in version 1.6.0: Added timeout and cwd arguments.
- Parameters
command (str,list) – command to be issued
default (object) – response if the query fails
ignore_exit_status (bool) – reports failure if our command’s exit status was non-zero
timeout (float) – maximum seconds to wait, blocks indefinitely if None
env (dict) – environment variables
- Returns
list with the lines of output from the command
- Raises
CallError if this fails and no default was provided
CallTimeoutError if the timeout is reached without a default
-
stem.util.system.
get_process_name
()[source]¶ Provides the present name of our process.
- Returns
str with the present name of our process
-
stem.util.system.
set_process_name
(process_name)[source]¶ Renames our current process from “python <args>” to a custom name. This is best-effort, not necessarily working on all platforms.
Note: This might have issues on FreeBSD (ticket 9804).
- Parameters
process_name (str) – new name for our process
-
stem.util.system.
get_name_by_pid
(pid)¶ Attempts to determine the name a given process is running under (not including arguments). This uses…
1. Information from /proc 2. ps -p <pid> -o command
- Parameters
pid (int) – process id of the process to be queried
- Returns
str with the process name, None if it can’t be determined
-
stem.util.system.
get_pid_by_name
(process_name, multiple=False)¶ Attempts to determine the process id for a running process, using…
1. pgrep -x <name> 2. pidof <name> 3. ps -o pid -C <name> (linux) ps axc | egrep " <name>$" (bsd) 4. lsof -tc <name> 5. tasklist | str <name>.exe
- Parameters
process_name (str) – process name for which to fetch the pid
multiple (bool) – provides a list of all pids if True, otherwise results with multiple processes are discarded
- Returns
Response depends upon the ‘multiple’ argument as follows…
if False then this provides an int with the process id or None if it can’t be determined
if True then this provides a list of all int process ids, and an empty list if it can’t be determined
-
stem.util.system.
get_pid_by_port
(port)¶ Attempts to determine the process id for a process with the given port, using…
1. netstat -npltu | grep 127.0.0.1:<port> 2. sockstat -4l -P tcp -p <port> 3. lsof -wnP -iTCP -sTCP:LISTEN | grep ":<port>"
Most queries limit results to listening TCP connections. This function likely won’t work on Mac OSX.
- Parameters
port (int) – port where the process we’re looking for is listening
- Returns
int with the process id, None if it can’t be determined
-
stem.util.system.
get_pid_by_open_file
(path)¶ Attempts to determine the process id for a process with the given open file, using…
lsof -w <path>
- Parameters
path (str) – location of the socket file to query against
- Returns
int with the process id, None if it can’t be determined
-
stem.util.system.
get_cwd
(pid)¶ Provides the working directory of the given process.
- Parameters
pid (int) – process id of the process to be queried
- Returns
str with the absolute path for the process’ present working directory, None if it can’t be determined
-
stem.util.system.
get_user
(pid)¶ Provides the user a process is running under.
- Parameters
pid (int) – process id of the process to be queried
- Returns
str with the username a process is running under, None if it can’t be determined
-
stem.util.system.
get_start_time
(pid)¶ Provides the unix timestamp when the given process started.
- Parameters
pid (int) – process id of the process to be queried
- Returns
float for the unix timestamp when the process began, None if it can’t be determined
-
stem.util.system.
get_bsd_jail_id
(pid)¶ Gets the jail id for a process. These seem to only exist for FreeBSD (this style for jails does not exist on Linux, OSX, or OpenBSD).
- Parameters
pid (int) – process id of the jail id to be queried
- Returns
int for the jail id, zero if this can’t be determined
-
stem.util.system.
get_bsd_jail_path
(jid)¶ Provides the path of the given FreeBSD jail.
- Parameters
jid (int) – jail id to be queried
- Returns
str of the path prefix, None if this can’t be determined