class Net::SCP

Net::SCP implements the SCP (Secure CoPy) client protocol, allowing Ruby programs to securely and programmatically transfer individual files or entire directory trees to and from remote servers. It provides support for multiple simultaneous SCP copies working in parallel over the same connection, as well as for synchronous, serial copies.

Basic usage:

require 'net/scp'

Net::SCP.start("remote.host", "username", :password => "passwd") do |scp|
  # synchronous (blocking) upload; call blocks until upload completes
  scp.upload! "/local/path", "/remote/path"

  # asynchronous upload; call returns immediately and requires SSH
  # event loop to run
  channel = scp.upload("/local/path", "/remote/path")
  channel.wait
end

Net::SCP also provides an open-uri tie-in, so you can use the Kernel#open method to open and read a remote file:

# if you just want to parse SCP URL's:
require 'uri/scp'
url = URI.parse("scp://user@remote.host/path/to/file")

# if you want to read from a URL voa SCP:
require 'uri/open-scp'
puts open("scp://user@remote.host/path/to/file").read

Lastly, Net::SCP adds a method to the Net::SSH::Connection::Session class, allowing you to easily grab a Net::SCP reference from an existing Net::SSH session:

require 'net/ssh'
require 'net/scp'

Net::SSH.start("remote.host", "username", :password => "passwd") do |ssh|
  ssh.scp.download! "/remote/path", "/local/path"
end

Progress Reporting

By default, uploading and downloading proceed silently, without any outward indication of their progress. For long running uploads or downloads (and especially in interactive environments) it is desirable to report to the user the progress of the current operation.

To receive progress reports for the current operation, just pass a block to upload or download (or one of their variants):

scp.upload!("/path/to/local", "/path/to/remote") do |ch, name, sent, total|
  puts "#{name}: #{sent}/#{total}"
end

Whenever a new chunk of data is recieved for or sent to a file, the callback will be invoked, indicating the name of the file (local for downloads, remote for uploads), the number of bytes that have been sent or received so far for the file, and the size of the file.

Attributes

session[R]

The underlying Net::SSH session that acts as transport for the SCP packets.

Public Class Methods

download!(host, username, remote, local=nil, options={}, &progress) click to toggle source

Starts up a new SSH connection using the host and username parameters, instantiates a new SCP session on top of it, and then begins a download from remote to local. If the options hash includes an :ssh key, the value for that will be passed to the SSH connection as options (e.g., to set the password, etc.). All other options are passed to the download! method. If a block is given, it will be used to report progress (see “Progress Reporting”, under Net::SCP).

# File lib/net/scp.rb, line 238
def self.download!(host, username, remote, local=nil, options={}, &progress)
  options = options.dup
  start(host, username, options.delete(:ssh) || {}) do |scp|
    return scp.download!(remote, local, options, &progress)
  end
end
new(session) click to toggle source

Creates a new Net::SCP session on top of the given Net::SSH session object.

# File lib/net/scp.rb, line 251
def initialize(session)
  @session = session
  self.logger = session.logger
end
start(host, username, options={}) { |scp| ... } click to toggle source

Starts up a new SSH connection and instantiates a new SCP session on top of it. If a block is given, the SCP session is yielded, and the SSH session is closed automatically when the block terminates. If no block is given, the SCP session is returned.

# File lib/net/scp.rb, line 201
def self.start(host, username, options={})
  session = Net::SSH.start(host, username, options)
  scp = new(session)

  if block_given?
    begin
      yield scp
      session.loop
    ensure
      session.close
    end
  else
    return scp
  end
end
upload!(host, username, local, remote, options={}, &progress) click to toggle source

Starts up a new SSH connection using the host and username parameters, instantiates a new SCP session on top of it, and then begins an upload from local to remote. If the options hash includes an :ssh key, the value for that will be passed to the SSH connection as options (e.g., to set the password, etc.). All other options are passed to the upload! method. If a block is given, it will be used to report progress (see “Progress Reporting”, under Net::SCP).

# File lib/net/scp.rb, line 224
def self.upload!(host, username, local, remote, options={}, &progress)
  options = options.dup
  start(host, username, options.delete(:ssh) || {}) do |scp|
    scp.upload!(local, remote, options, &progress)
  end
end

Public Instance Methods

download(remote, local, options={}, &progress) click to toggle source

Inititiate a synchronous (non-blocking) download from remote to local. The following options are recognized:

  • :recursive - the remote parameter refers to a remote directory, which should be downloaded to a new directory named local on the local machine.

  • :preserve - the atime and mtime of the file should be preserved.

  • :verbose - the process should result in verbose output on the server end (useful for debugging).

This method will return immediately, returning the Net::SSH::Connection::Channel object that will support the download. To wait for the download to finish, you can either call the wait method on the channel, or otherwise run the Net::SSH event loop until the channel's active? method returns false.

channel = scp.download("/remote/path", "/local/path")
channel.wait
# File lib/net/scp.rb, line 304
def download(remote, local, options={}, &progress)
  start_command(:download, local, remote, options, &progress)
end
download!(remote, local=nil, options={}, &progress) click to toggle source

Same as download, but blocks until the download finishes. Identical to calling download and then calling the wait method on the channel object that is returned.

scp.download!("/remote/path", "/local/path")

If local is nil, and the download is not recursive (e.g., it is downloading only a single file), the file will be downloaded to an in-memory buffer and the resulting string returned.

data = download!("/remote/path")
# File lib/net/scp.rb, line 319
def download!(remote, local=nil, options={}, &progress)
  destination = local ? local : StringIO.new
  download(remote, destination, options, &progress).wait
  local ? true : destination.string
end
upload(local, remote, options={}, &progress) click to toggle source

Inititiate a synchronous (non-blocking) upload from local to remote. The following options are recognized:

  • :recursive - the local parameter refers to a local directory, which should be uploaded to a new directory named remote on the remote server.

  • :preserve - the atime and mtime of the file should be preserved.

  • :verbose - the process should result in verbose output on the server end (useful for debugging).

  • :chunk_size - the size of each “chunk” that should be sent. Defaults to 2048. Changing this value may improve throughput at the expense of decreasing interactivity.

This method will return immediately, returning the Net::SSH::Connection::Channel object that will support the upload. To wait for the upload to finish, you can either call the wait method on the channel, or otherwise run the Net::SSH event loop until the channel's active? method returns false.

channel = scp.upload("/local/path", "/remote/path")
channel.wait
# File lib/net/scp.rb, line 276
def upload(local, remote, options={}, &progress)
  start_command(:upload, local, remote, options, &progress)
end
upload!(local, remote, options={}, &progress) click to toggle source

Same as upload, but blocks until the upload finishes. Identical to calling upload and then calling the wait method on the channel object that is returned. The return value is not defined.

# File lib/net/scp.rb, line 283
def upload!(local, remote, options={}, &progress)
  upload(local, remote, options, &progress).wait
end

Private Instance Methods

await_response(channel, next_state) click to toggle source

Causes the state machine to enter the “await response” state, where things just pause until the server replies with a 0 (see await_response_state), at which point the state machine will pick up at next_state and continue processing.

# File lib/net/scp.rb, line 382
def await_response(channel, next_state)
  channel[:state] = :await_response
  channel[:next ] = next_state.to_sym
  # check right away, to see if the response is immediately available
  await_response_state(channel)
end
await_response_state(channel) click to toggle source

The action invoked while the state machine remains in the “await response” state. As long as there is no data ready to process, the machine will remain in this state. As soon as the server replies with an integer 0 as the only byte, the state machine is kicked into the next state (see await_response). If the response is not a 0, an exception is raised.

# File lib/net/scp.rb, line 395
def await_response_state(channel)
  return if channel[:buffer].available == 0
  c = channel[:buffer].read_byte
  raise "#{c.chr}#{channel[:buffer].read}" if c != 0
  channel[:next], channel[:state] = nil, channel[:next]
  send("#{channel[:state]}_state", channel)
end
finish_state(channel) click to toggle source

The action invoked when the state machine is in the “finish” state. It just tells the server not to expect any more data from this end of the pipe, and allows the pipe to drain until the server closes it.

# File lib/net/scp.rb, line 406
def finish_state(channel)
  channel.eof!
end
progress_callback(channel, name, sent, total) click to toggle source

Invoked to report progress back to the client. If a callback was not set, this does nothing.

# File lib/net/scp.rb, line 412
def progress_callback(channel, name, sent, total)
  channel[:callback].call(channel, name, sent, total) if channel[:callback]
end
scp_command(mode, options) click to toggle source

Constructs the scp command line needed to initiate and SCP session for the given mode (:upload or :download) and with the given options (:verbose, :recursive, :preserve). Returns the command-line as a string, ready to execute.

# File lib/net/scp.rb, line 331
def scp_command(mode, options)
  command = "scp "
  command << (mode == :upload ? "-t" : "-f")
  command << " -v" if options[:verbose]
  command << " -r" if options[:recursive]
  command << " -p" if options[:preserve]
  command
end
shellescape(path) click to toggle source

Imported from ruby 1.9.2 shellwords.rb

# File lib/net/scp.rb, line 417
def shellescape(path)
  # Convert path to a string if it isn't already one.
  str = path.to_s

  # ruby 1.8.7+ implements String#shellescape
  return str.shellescape if str.respond_to? :shellescape

  # An empty argument will be skipped, so return empty quotes.
  return "''" if str.empty?

  str = str.dup

  # Process as a single byte sequence because not all shell
  # implementations are multibyte aware.
  str.gsub!(/([^A-Za-z0-9_\-.,:\/@\n])/n, "\\\\\\1")

  # A LF cannot be escaped with a backslash because a backslash + LF
  # combo is regarded as line continuation and simply ignored.
  str.gsub!(/\n/, "'\n'")

  return str
end
start_command(mode, local, remote, options={}, &callback) click to toggle source

Opens a new SSH channel and executes the necessary SCP command over it (see scp_command). It then sets up the necessary callbacks, and sets up a state machine to use to process the upload or download. (See Net::SCP::Upload and Net::SCP::Download).

# File lib/net/scp.rb, line 344
def start_command(mode, local, remote, options={}, &callback)
  session.open_channel do |channel|
  
    if options[:shell]
      escaped_file = shellescape(remote).gsub(/'/) { |m| "'\\''" }
      command = "#{options[:shell]} -c '#{scp_command(mode, options)} #{escaped_file}'"
    else
      command = "#{scp_command(mode, options)} #{shellescape remote}"
    end

    channel.exec(command) do |ch, success|
      if success
        channel[:local   ] = local
        channel[:remote  ] = remote
        channel[:options ] = options.dup
        channel[:callback] = callback
        channel[:buffer  ] = Net::SSH::Buffer.new
        channel[:state   ] = "#{mode}_start"
        channel[:stack   ] = []
        channel[:error_string] = ''

        channel.on_close                  { |ch| send("#{channel[:state]}_state", channel); raise Net::SCP::Error, "SCP did not finish successfully (#{channel[:exit]}): #{channel[:error_string]}" if channel[:exit] != 0 }
        channel.on_data                   { |ch, data| channel[:buffer].append(data) }
        channel.on_extended_data          { |ch, type, data| debug { data.chomp } }
        channel.on_request("exit-status") { |ch, data| channel[:exit] = data.read_long }
        channel.on_process                { send("#{channel[:state]}_state", channel) }
      else
        channel.close
        raise Net::SCP::Error, "could not exec scp on the remote host"
      end
    end
  end
end