Utils for streaming large files (S3, HDFS, gzip, bz2, and more)
Description
smart_open is a Python 3 library for efficient streaming of very large files
from/to storages such as S3, GCS, Azure Blob Storage, HDFS, WebHDFS, HTTP,
HTTPS, SFTP, or local filesystem. It supports transparent, on-the-fly
(de-)compression for a variety of different formats.
smart_open is a drop-in replacement for Pythons built-in open(): it can do
anything open can (100% compatible, falls back to native open wherever
possible), plus lots of nifty extra stuff on top.
