Skip to content

Adds ssm:copy command to allow for scp over ssm #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Adds ssm:copy command to allow for scp over ssm #13

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions app/Commands/AwsSsmConnect.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ protected function sendReRunHelper($rebuildOptions): void
{
$this->info("You can run this command again without having to go through options using this:");
$this->info(' ');
$this->comment("netsells aws:ssm:connect " . implode(' ', $rebuildOptions));
$this->comment("netsells " . $this->signature . " " . implode(' ', $rebuildOptions));
$this->info(' ');
}

Expand Down Expand Up @@ -189,7 +189,7 @@ protected function askForInstanceId()
return $this->menu("Choose an instance to connect to...", $instances->toArray())->open();
}

private function generateTempSshKey()
protected function generateTempSshKey()
{
$requiredBinaries = ['aws', 'ssh', 'ssh-keygen'];

Expand Down Expand Up @@ -227,7 +227,7 @@ private function generateTempSshKey()
return trim(file_get_contents($pubKeyName));
}

private function generateRemoteCommand($username, $key)
protected function generateRemoteCommand($username, $key)
{
// Borrowed from https://github.com/elpy1/ssh-over-ssm/blob/master/ssh-ssm.sh#L10
return trim(<<<EOF
Expand Down
161 changes: 161 additions & 0 deletions app/Commands/AwsSsmCopy.php
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
<?php

namespace App\Commands;

use App\Exceptions\ProcessFailed;
use Symfony\Component\Process\Process;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Output\OutputInterface;

class AwsSsmCopy extends AwsSsmConnect
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see few improvements that could be made.

To reduce the copy-pasta, I think the main AwsSssConnect command could use a template method pattern. We can have a protected method "composeProcessComand" which returns the "command array" (passed to the withCommand method) that is called from within handle method. This way AwsSsmCopy could just override the configure + composeProcessCommand with the common code staying in one place. However since we plan to rewrite in Go anyway, I think it's less of a concern and we can focus on making the command user friendly, which brings me to the next point.

I think this command should be as close to actual scp as possible to make it as intuitive (and familiar) as can be. Instead of asking for direction, the direction would be from left to right, source to destination, just like scp and cp. Given that it uses scp underneath, I think we could just proxy arguments and options down to the scp command. In my mind, we can use it just like:

## copy local to remote 
netsells aws:ssm:copy ~/Desktop/hello.txt ec2-user@i-e123412:/var/www/

## copy remote to local
netsells aws:ssm:copy ec2-user@i-e123412:/var/www/hello.txt ~/Desktop/

If we proxy the arguments and options, the above should "just work". It will also support options such as '-r' for recursive copying - so bigs wins straightaway.

The only problem remaining, is that we need to know the username and host/instance. The username and instance should be optional, allowing the user to pick them from a list (as is currently). In order to do that, I think we can parse the arguments using parse_url and determine the source/destination + user/host. Few examples:

parse_url('file://ec2-user@i-e123412:/var/www/')
=> [
     "scheme" => "file",
     "host" => "i-e123412",
     "user" => "ec2-user",
     "path" => "/var/www/",
   ]

parse_url('file://:/var/www/')
=> false

parse_url('file://~/Desktop/hello.txt')
=> [
     "scheme" => "file",
     "host" => "~",
     "path" => "/Desktop/hello.txt",
   ]

Algorithm:

  • take argument, prepend 'file://' to it and pass to parse_url
  • if result is false, we interactively ask for username + instance, and then know that full path is $username@$hostname$argument
  • if result has user + host keys, we set username + instance to these, and proxy the argument as is
  • otherwise we just proxy the argument as is
  • once we have gone through all arguments, we check if username + instance is set - if not, we error out as we can't determine where to connect to

The above will allow for doing this, when we don't know the server:

## copy local to remote 
netsells aws:ssm:copy ~/Desktop/hello.txt :/var/www/

## copy remote to local
netsells aws:ssm:copy :/var/www/hello.txt ~/Desktop/

And of course it will allow for the first version, where the username + host is explicitly passed in.

Perhaps we could do the parsing in a different way, that's just an idea I wanted to put forward. I think starting the path with just colon makes sense to denote remote server, and still is very similar to usual scp.

Please let me know what you think @spamoom

{
/**
* The signature of the command.
*
* @var string
*/
protected $signature = 'aws:ssm:copy';

/**
* The description of the command.
*
* @var string
*/
protected $description = 'Copy files from/to a server over SSM';

public function configure()
{
$this->setDefinition(array_merge([
new InputOption('instance-id', null, InputOption::VALUE_OPTIONAL, 'The instance ID to connect to'),
new InputOption('username', null, InputOption::VALUE_OPTIONAL, 'The username connect with'),
new InputOption('direction', null, InputOption::VALUE_OPTIONAL, 'Direction (Up/Down)'),
new InputOption('local-path', null, InputOption::VALUE_OPTIONAL, 'The path of the other server (local or remote). Can be a file or folder'),
new InputOption('remote-path', null, InputOption::VALUE_OPTIONAL, 'The path of the server. Can be a file or folder'),
new InputOption('other-server', null, InputOption::VALUE_OPTIONAL, 'Left blank, the other server is the local machine. Otherwise, specify user@host'),
], $this->helpers->aws()->commonConsoleOptions()));
}

/**
* Execute the console command.
*
* @return mixed
*/
public function handle()
{
$requiredBinaries = ['aws', 'ssh'];

if ($this->helpers->checks()->checkAndReportMissingBinaries($requiredBinaries)) {
return 1;
}

$rebuildOptions = [];

$instanceId = $this->option('instance-id') ?: $this->askForInstanceId();
$username = $this->option('username') ?: $this->askForUsername();

if (!$instanceId) {
$this->error('No instance ID provided.');
return 1;
}

$rebuildOptions = $this->appendResolvedArgument($rebuildOptions, 'username', $username);
$rebuildOptions = $this->appendResolvedArgument($rebuildOptions, 'instance-id', $instanceId);
$rebuildOptions = $this->appendResolvedArgument($rebuildOptions, 'aws-profile');
$rebuildOptions = $this->appendResolvedArgument($rebuildOptions, 'aws-region');

$key = $this->generateTempSshKey();
$command = $this->generateRemoteCommand($username, $key);

$this->info("Sending a temporary SSH key to the server...", OutputInterface::VERBOSITY_VERBOSE);
if (!$this->helpers->aws()->ssm()->sendRemoteCommand($instanceId, $command)) {
$this->error('Failed to send SSH key to server');
return 1;
}

$sessionCommand = $this->helpers->aws()->ssm()->startSessionProcess($instanceId);
$sessionCommandString = implode(' ', $sessionCommand->getArguments());

$options = [
'-o', 'IdentityFile ~/.ssh/netsells-cli-ssm-ssh-tmp',
'-o', 'IdentitiesOnly yes',
'-o', 'GSSAPIAuthentication no',
'-o', 'PasswordAuthentication no',
'-o', "ProxyCommand {$sessionCommandString}",
];

$direction = trim($this->option('direction')) ?: $this->askForDirection();

if (!$this->validateDirection($direction)) {
$this->error('Invalid direction');
}

$rebuildOptions = $this->appendResolvedArgument($rebuildOptions, 'direction', $direction);

$otherServer = trim($this->option('other-server')) ?: $this->askForOtherServer();

$localPath = trim($this->option('local-path')) ?: $this->askForLocalPath($otherServer);
$remotePath = trim($this->option('remote-path')) ?: $this->askForRemotePath();

$rebuildOptions = $this->appendResolvedArgument($rebuildOptions, 'local-path', $localPath);
$rebuildOptions = $this->appendResolvedArgument($rebuildOptions, 'remote-path', $remotePath);
$rebuildOptions = $this->appendResolvedArgument($rebuildOptions, 'other-server', $otherServer);

$this->sendReRunHelper($rebuildOptions);

$localOption = ($otherServer !== '__localhost__' ? "{$otherServer}:" : null) . $localPath;
$remoteOption = sprintf("%s@%s", $username, $instanceId) . ":{$remotePath}";

$directionOptions = ($direction == 'up') ? [$localOption, $remoteOption] : [$remoteOption, $localOption];

try {
$this->helpers->process()->withCommand(array_merge(
[
'scp',
],
$options,
$directionOptions,
))
->withTimeout(null)
->withProcessModifications(function ($process) {
$process->setTty(Process::isTtySupported());
$process->setIdleTimeout(null);
})
->run();
} catch (ProcessFailed $e) {
$this->info(' ');
$this->error("SCP command exited with an exit code of " . $e->getCode());
}
}

protected function askForDirection()
{
return $this->menu("Which direction are you sending the files?", [
'up' => 'Upstream',
'down' => 'Downstream',
])->open();
}

protected function validateDirection($direction): bool
{
return in_array($direction, ['up', 'down']);
}

protected function askForLocalPath($otherServer)
{
if ($otherServer == '__localhost__') {
return $this->ask("What is the path of the file/folder on your computer?");
}

return $this->ask("What is the path of the file/folder on the other server? ({$otherServer})");
}

protected function askForRemotePath()
{
return $this->ask("What is the path of the file/folder on the remote server?");
}

protected function askForOtherServer()
{
return $this->ask("What is the path of the other server? Should be in the format username@hostname. Leave blank for this computer", '__localhost__');
}
}
2 changes: 1 addition & 1 deletion app/Commands/Console/InputOption.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ public function getDefault()
$constantName = sprintf("%s::%s", NetsellsFile::class, $this->netsellsFilePrefix . $keyName);

if (defined($constantName)) {
return (new NetsellsFile())->get(constant($constantName));
return (new NetsellsFile())->get(constant($constantName), parent::getDefault());
}

return parent::getDefault();
Expand Down
2 changes: 1 addition & 1 deletion app/Helpers/Aws/Ec2.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ public function listInstances($query): ?Collection
$processOutput = $this->aws->newProcess($commandOptions)
->run();
} catch (ProcessFailed $e) {
$this->command->error("Unable to list ec2 instances");
$this->aws->getCommand()->error("Unable to list ec2 instances");
return null;
}

Expand Down