Run commands over SSH!

Sometimes I’ll need to run a few command on a remote machine. In the past, I would SSH into the machine and I would started making changes. But this is not ideal since this isn’t repeatable. If you’ve run into this problem, I have an alternative! You can execute a shell script from your local system and have it run commands on the remote system using SSH!

Please be aware that there are better tools – Ansible, Terraform, CloudFormation, etc. But those are heavy tools.

In this example, I have some configuration files that need to be copied to the remote system, and then I need to execute a couple commands. I also wanted the script to be checked into version control.

The first half of the script copies files, using scp, from my local machine to the remote machine. The second half of the script (starting on line 12) runs commands from the remote machine. The files are moved to the correct locations and Nginx and Haproxy are restarted.

#!/bin/bash

PEM=id_rsa.pub
HOST=ec2-54-234-130-49.compute-1.amazonaws.com

scp -i ~/.ssh/$PEM ./surrogate_pop.conf ubuntu@$HOST:/tmp
scp -i ~/.ssh/$PEM ./haproxy.cfg ubuntu@$HOST:/tmp
scp -i ~/.ssh/$PEM ./traffic_cop.lua ubuntu@$HOST:/tmp
scp -i ~/.ssh/$PEM ./allowed_domains.lua ubuntu@$HOST:/tmp

## These are executed on the remote host
ssh -i ~/.ssh/$PEM ubuntu@$HOST 'bash -s' <<EOF
sudo mv /tmp/traffic_cop.lua /usr/share/nginx/traffic_cop.lua
sudo mv /tmp/allowed_domains.lua /usr/share/nginx/allowed_domains.lua
sudo mv /tmp/surrogate_pop.conf /etc/nginx/sites-enabled/surrogate_pop.conf
sudo service nginx restart

sudo mv /tmp/haproxy.cfg /etc/haproxy/haproxy.cfg
sudo service haproxy restart
EOF

Cloudformation ApiGateway and Lambda

Recently, I’ve been excited by serverless technology. I began using the serverless framework for code boilerplate and deployment. After some time using the framework, I began feeling pain. Serverless is an excellent project, but it’s moving very fast. For example, The framework uses cloudformation for resource dependencies such as dynamoDb, ApiGateway, roles and permissions (to name a few). Cloudformation is also moving very fast. Support for ApiGateway was added to cloudformation on April 18th, 2016. As new features are added to cloudformation, you’ll be stuck waiting for serverless to implement features for feature parity. I’ve started using cloudformation direclty and relying on bash scripts for deployment. I’m quite happy with the results.

Cloudformation stack

Once we have a cloudformation template, the AWS cli provides us with everthing we need. Using the AWS CLI we can create the stack like so.

$ aws cloudformation create-stack 
	--stack-name hello-world 
	--template-body file://./cloudformation.json 
	--capabilities CAPABILITY_IAM && 
	aws cloudformation wait stack-create-complete 
		--stack-name hello-world

The first command fires off an async create request to AWS. The second command tells our shell to wait for stack creation to complete.

After that’s complete, we’ll have created a few resources in AWS. =) Next, we’ll need a way to deploy.

Deployment

We have a few tasks for a complete deployment. We should seperate out the lambda depoyment and the ApiGateway deployment, but in this case I did not.

  • Update Lambda Code - Install any dependencies, zip our code, and upload it.
  • Publish a Version - From the latest version, It will tag a copy.
  • Update the alias - Our lambda is pointed to an alias. This will point the lambda to our new version.
  • deploy ApiGateway - Any changes we make to ApiGateway requires a deploy.

The script takes two args. The api-gateway-id and the function-name.

$ ./deploy.sh abc123 hello-word
#!/bin/bash

apiId=$1
functionName=$2
profile=personal


YELLOW='\033[0;33m'
WHITE='\033[0m' # No Color

function zipLambda {
  say "Zipping files." && 
  rm -rf target && 
  mkdir -p target && 
  cp -r *.js package.json target/ && 
  pushd target && 
  npm install --production && 
  zip -r "${functionName}.zip" . && 
  popd
}

function say {
  printf "\n${YELLOW} $@ ${WHITE}\n"
}

function updateLambdaCode {
  say "Uploading new lambda code." && 
  aws lambda update-function-code --function-name $functionName --zip-file "fileb://target/${functionName}.zip" --profile $profile
}

function publishVersion {
  say "Publishing a new version." && 
  aws lambda publish-version --function-name $functionName --profile $profile
}

function updateAlias {
  version=$(aws lambda list-versions-by-function --function-name $functionName --profile personal | grep Version | tail -n 1 | cut -d '"' -f 4) && 
  say "Updating the alias to version ${version}." && 
  aws lambda update-alias --function-name $functionName --function-version $version --name prod --profile $profile
}
function deployApiGatway {
  say "Deploying to Api Gateway." && 
  aws apigateway create-deployment --rest-api-id $apiId --stage-name v1 --profile $profile
}
printf "\n🚀🚀🚀 SHIP IT!!! 🚀🚀🚀 \n\n"
zipLambda && 
  updateLambdaCode && 
  publishVersion && 
  updateAlias && 
  deployApiGatway

ActiveModel::Model

Rails 4 brought us ActiveModel::Model. It provides a light weight interface that’s similar to an ActiveRecord::Base model.

for example, I can create a Person class like so.

class Person
  include ActiveModel::Model
  attr_accessor :name, :age

  validates :name, true

  def save
    ## Do cool stuff here...
  end
end
Loading development environment (Rails 4.2.1)
irb(main):001:0> p = Person.new age: 21
=> #<Person:0x007fc6367193b0 @age=21>
irb(main):002:0> p.valid?
=> false
irb(main):003:0> p.errors
=> #<ActiveModel::Errors:0x007fc638000a68 @base=#<Person:0x007fc6367193b0 @age=21, @validation_context=nil, @errors=#<ActiveModel::Errors:0x007fc638000a68 ...>>, @messages={:name=>["can't be blank"]}>
irb(main):004:0>

This is great for instances where you don’t need a full database backed Active Record model. I’ve used them for form objects and in controllers where I have complex logic.

You can think of these as higher level abstractions above your ActiveRecord classes. Also, be conscious of the dependancy direction. An ActiveModel model can depend on an ActiveRecord model but your ActiveRecord models shouldn’t depend on an ActiveModel model.


Here’s a more involved example. Lets say I have 2 ActiveRecord classes Org and User

class Org < ActiveRecord::Base
  validates :name, presence: true
end
class User < ActiveRecord::Base
  validates :first, :last, presence: true
end

Now I’ll create an ActiveModel model (non database)

Notice the validates_each method… Its going to check each of the ActiveRecord objects and let them raise up any errors to the Signup class.

class Signup
  include ActiveModel::Model
  attr_accessor :first, :last, :name

  validates_each :user, :org do |record, attr, value|
    unless value.valid?
      value.errors.each { |k,v| record.errors.add(k.to_sym, v) }
    end
  end

  ## must return boolean
  def save
    if valid?
      org.save && user.save
    else
      false
    end
  end

  private

    def org
      @org ||= Org.new(name: name)
    end

    def user
      @user ||= User.new(first: first, last: last)
    end
end

Awesome right!!

So why do all this? Well, the single responsibilty states that every class should have responsibility over a single part of the functionality provided by the software, and that responsibility should be entirely encapsulated by the class. ActiveRecord is responsible for persistence to the database. This will keep our classes with a narrow focus and allow us to refactor and create more use cases in the future. I think it’s a win. I find this strategy is generally good for one directional workflows such as signup or in a shopping app cart checkout.

rails configuration

Occasionally I’ll see things like this in a code base.

def api_host
  if Rails.production?
    "http://prod.fake.api.url"
  else
    "http://stag.fake.api.url"
  end
end

I try to avoid writting methods like this. Rails provides a nice way to set environment specific variables.

http://guides.rubyonrails.org/configuring.html#custom-configuration

config/environments/staging.rb

config.api_host = "http://stag.fake.api.url"

config/environments/production.rb

config.api_host = "http://prod.fake.api.url"

So now you can refactor the method to this.

def api_host
  Rails.configuration.api_host
end

Crypto

I’ve been playing around with Public-key cryptography or asymmetric cryptography. With revaltions from Eric Snowden on programs like PRISM, I feel that encryption is more important than ever.

From Wikipedia

Public-key cryptography is a class of cryptographic algorithms which requires two separate keys, one of which is secret (or private) and one of which is public. Although different, the two parts of this key pair are mathematically linked. The public key is used to encrypt plaintext or to verify a digital signature; whereas the private key is used to decrypt ciphertext or to create a digital signature.

There are two main uses for public key crypto.

  1. Public key encryption. A message is encrypted with a recipient’s public key. The message can only be unencrypted by the holder of the private key.

  2. Digital Signature. A message is signed with a the senders private key to generate a cryptographic hash. Using the public key, one can verify the message was indeed sent with the coresponding private key and can ensure the message has not been altered.

Keys

First, let’s generate some keys. From the private key we will also generate the public key. Remember the private and public key are mathematically linked. This is important.

[1] pry(main)> require "openssl"
=> true
[2] pry(main)> private_key = OpenSSL::PKey::RSA.new(2048)
=> #<OpenSSL::PKey::RSA:0x007ff03b379c30>
[3] pry(main)> public_key = private_key.public_key
=> #<OpenSSL::PKey::RSA:0x007ff03c3330b8>

Great. Now we have a key pair. We can write these keys to disk if we like.

## write the private key to disk.
File.open("~/.ssh/private_key", "w+") do |f|
  f.write private_key.to_pem
end
# write the public key to disk.
File.open("~/.ssh/public_key.pem", "w+") do |f|
  f.write public_key.to_pem
end

We can read the key in from key like so.

key = OpenSSL::PKey::RSA.new(File.read("~/.ssh/private_key"))

Public key enctryption

We’ll be encrypting a message with the public key. So it can only be unenctrypted by the owner of the private key.

## Use the public key to encrypt a message
[4] pry(main)> encrypted_data = public_key.public_encrypt("Some private data is here.")
=> "›Ò"lûú¢Ì;ŸÂrúˆ”\ÛZY–4ãiÛµ´<Žì¸«"aáàŒb쮆À!È©ÞBe^Ìe܁ܘXÖ"bUJ°Bí–Ÿ}÷IýŸôQ§HœÅaá¤kxô6A% ëë: Ñ¢ÿ¤$½PŸRéӟ\r>Ž5ÙèYde`Žà½å\öƒÔmÍJš*¦õS9ÒMÕ 7’Îð¹<¯eš®Ž`­W“‘;HZëØÿÞmò*\tL2â`KÈ)O†Ú¯k4Ð\rNó%}êMheßÆåª$V¾~ŽLvœÇ´o¡3âÞât!]-Ȉ¯ãÕý`TÙÍsÿ®(<½¹"

This will result in some unreadable binary data. Next we’ll decrypt this data with the private key.

## Use the private key to decrypt
[5] pry(main)> private_key.private_decrypt(enc)
=> "Some private data is here."

Nice!

The public key can encrypt but cannot decrypt the message. This enctryption only works in one direction. Only the private key can decrypt that message. You can also encrypt messages with a private key. Those messages can be unenctrypted with the public key and the private key.

Asymmetric public/private key encryption is slow and victim to attack in cases where it is used without padding or directly to encrypt larger chunks of data. Typical use cases for RSA encryption involve “wrapping” a symmetric key with the public key of the recipient who would “unwrap” that symmetric key again using their private key.

NOTE - The enctrypted messages are in binary format. If we want to encode the data for transmission accoss the net, we’ll want to use something like base64.

[6] pry(main)> require "base64"
=> true
[7] pry(main)> enc_string = Base64.encode64(encrypted)
=> "iRBt6WHFYP2sn2Qv+qs16js/EJqBGaTWyxUH7iI/aj3UEw1oUHHbrFs/705W\nP+8dJ77p5gAaBpS/spCYLu/strU3uN06DTOh3neTcyDQrpIL5Zqs0Gl6/76m\nOQFGi18khnwWPAyW4+uVcZiQmZU9M0tlWywwNlkVoKAwkFlwkYF07YZazfCY\nMrAoQ6nusfjqjfU7HQeQKSCnMrBkzInsqan0PUm+UuGCMbxpQMdPA1de2nHB\neMs7OR7Pd5q5T93z240Iacjtwo/CV3Tcr1EyrfCcx05Jp4FKi9DPJf33asPx\noJ7J2XNa3QXqEbisMGT/b+6QZDm/LbfZXKuCIDYOjA==\n"

This works for small messages but if you try and encrpt a string larget than your key… you’ll hit an errors. You could break up the string into smaller chunks and encrypt each chunk individually. But this is not secure. This is what cypher block chaining is for. Below is an example.

cipher      = OpenSSL::Cipher.new 'AES-128-CBC'
cipher.encrypt
iv          = cipher.random_iv
pwd         = 'tRiSsmiTp'
salt        = OpenSSL::Random.random_bytes 16
iter        = 20000
key_len     = cipher.key_len
digest      = OpenSSL::Digest::SHA256.new

key = OpenSSL::PKCS5.pbkdf2_hmac(pwd, salt, iter, key_len, digest)
cipher.key = key

File.open("enc-text.txt",'w') do |enc|
  File.open("./plain.txt") do |f|
     loop do
       r = f.read(4096)
       break unless r
       enc << cipher.update(r)
     end
     enc << cipher.final
  end
end

Digital Signature

Asymmetric digital signatures is a great way to verify integrity and authenticity of data. Create a keypair, send the public key to your receivers, and use this method to create a digital signature. By combining the data and the public key, you can verify that the signature was created by the owner of the private key.

require "openssl"

data = "A small brown fox."


digest = OpenSSL::Digest::SHA256.new
# To list available digests:
#OpenSSL::Digest.constants

signature = private_key.sign(digest, data)

public_key.verify(digest, signature, data)
# => true
public_key.verify(digest, signature, data + "altered")
# => false
public_key.verify(digest, "altered" + signature, data)
# => false

More

Using openssl library directly we can encrypt a file.

openssl aes-256-cbc -a -e -in source-file.txt -out enc-file.txt -k private_key

And to decrypt the file.

openssl aes-256-cbc -d -a -in enc-file.txt -out unenc-file.txt -k private_key

http://ruby-doc.org/stdlib-2.0/libdoc/openssl/rdoc/OpenSSL.html#label-PBKDF2+Password-based+Encryption

Previous Next