Chef & Strainer and how my chef-repo was lacking.

Just to start I love Seths work, I know he gets pissed when people comment about his work this is not a comment on his work its a, I hope your google search comes up better then mine.

OK that said I spent a night learning how to get strainer running with travis-ci and read the instruction on the https://github.com/customink/strainer page and was like OK this should take no time at all. Turns out I was wrong. Looking back this should have taken me less then two minutes to figure out but let me be honest I blew more then a hour on this one.

For my testing I created a simple travis-chef-repo directory, dropped in a cookbook directory with the openssh cookbook and its dependency of apt and iptables. Followed the directions on https://github.com/customink/strainer to create a .travis.yml and a Strainer file. Commit it, and your off to the races. Not so quick.

First you need to make sure your Gemfile is correct in travis-chef-repo, here is my example:

jmiller11:travis-chef-repo jmiller$ cat Gemfile

source “https://rubygems.org”

gem ‘rake’
gem ‘chef’
gem ‘foodcritic’
gem ‘rspec’
gem ‘strainer’

Then create the Strainerfile in

jmiller11:travis-chef-repo jmiller$ cat Strainerfile

#
Strainerfile
knife test: bundle exec knife cookbook test $COOKBOOK
foodcritic: bundle exec foodcritic -f any $SANDBOX/$COOKBOOK

Then lets test it!

jmiller11:travistest-chef-repo jmiller$ bundle exec strainer test openssh
I could not detect if you were a chef-repo or a cookbook!
Strainer marked build OK

What the heck, why not? Maybe I have that command wrong lets try this one and maybe it will auto detect the cookbook.

jmiller11:travistest-chef-repo jmiller$ bundle exec strainer test
I could not detect if you were a chef-repo or a cookbook!
Strainer marked build OK

Or how about if we give it the path that must be it.

jmiller11:travistest-chef-repo jmiller$ bundle exec strainer test –cookbooks-path=./cookbooks/
I could not detect if you were a chef-repo or a cookbook!
Strainer marked build OK

Really, how do you tell if this is repo? Lets go look at the code:

https://github.com/customink/strainer/blob/master/lib/strainer/sandbox.rb#L54

else
Strainer.ui.warn “I could not detect if you were a chef-repo or a cookbook!”
@cookbooks = []
end

Umm yea that helps, let look at how we entered this if block:

https://github.com/customink/strainer/blob/master/lib/strainer/sandbox.rb#L54

if chef_repo?

OK so lets look at the chef_repo method:

https://github.com/customink/strainer/blob/master/lib/strainer/sandbox.rb#L244-L249

# Determines if the current project is a chef repo
#
# @return [Boolean]
# true if the current project is a chef repo, false otherwise
def chef_repo?
@_chef_repo ||= begin
chef_folders = %w(.chef certificates config cookbooks data_bags environments roles)
(root_folders & chef_folders).size > 2
end

What really, you need the following directories or are not a chef repo (.chef certificates config cookbooks data_bags environments roles)? At least its easy to fix.

mkdir .chef certificates config cookbooks data_bags environments roles

Lets test this sucker!

jmiller11:travistest-chef-repo jmiller$ bundle exec strainer test openssh
# Straining ‘openssh (v1.3.5)’
knife test | bundle exec knife cookbook test openssh
knife test | checking openssh
knife test | Running syntax check on openssh
knife test | Validating ruby files
knife test | Validating templates
knife test | SUCCESS!
foodcritic | bundle exec foodcritic -f any /Users/jmiller/Development/travistest-chef-repo/cookbooks/openssh
foodcritic | FC007: Ensure recipe dependencies are reflected in cookbook metadata: /Users/jmiller/Development/travistest-chef-repo/cookbooks/openssh/recipes/iptables.rb:20
foodcritic | Terminated with a non-zero exit status. Strainer assumes this is a failure.
foodcritic | FAILURE!
Strainer marked build as failure
jmiller11:travistest-chef-repo jmiller$

I win, lets add, commit, and push. Same “I could not detect if you were a chef-repo or a cookbook!” from travis … really whats going on here. Since I am not a git expert I realize git is not adding empty directories so lets add a file if one does not exist:

for i in certificates config .chef cookbooks data_bags environments roles; do touch $i/README.md;done

git add, commit, push and wait the 8 minutes for the gem install and success. I now see the same foodcritic errors I saw on the command line and have a failing build. Syntax check and lint tools running, time to move on.

Chef AWS

OK so you have your AWS account setup, your ready to launch a EC2 instance with Hosted Chef what do you need to know? I know this may seem simple to someone who has been doing it for a long time but it took me a few hours to figure what exactly I needed. Here is a few quick notes on what I found worked work me on my Macbook Air although it should work just fine on Linux also. I assume you have a working chef install and can upload cookbooks to your Hosted Chef server. Secondly I am using the chef-dk and had to install the knife-ec2 plugin. Assuming you have setup the chef-dk as they recommend all you should need to do is run the chef gem install command. This will install the ruby gems into your home directory at ~/.chefdk so you will not need sudo access.

jmiller11:fcs2-chef-repo jmiller$ ls -l ~/.chefdk/
total 0
drwxr-xr-x 3 jmiller staff 102 May 3 13:35 gem
jmiller11:fcs2-chef-repo jmiller$

Here is the command to install the plugin:

chef gem isntall knife-ec2

Append the following to your .chef/knife.rb

# AWS support
knife[:aws_access_key_id] = ENV[‘AWS_ACCESS_KEY_ID’]
knife[:aws_secret_access_key] = ENV[‘AWS_SECRET_ACCESS_KEY’]
# Optional if you’re using Amazon’s STS
#knife[:aws_session_token] = ENV[‘AWS_SESSION_TOKEN’]
knife[:aws_ssh_key_id] = ENV[‘AWS_MYPEM’]
knife[:region] = ENV[‘AWS_REGION’]
knife[:bootstrap_version]= ‘11.12.4-1’

Append the following to your ~/.bash_profile

AWS_ACCESS_KEY_ID=XXXXXXXXXXX
AWS_SECRET_ACCESS_KEY=XXXXXXXXXXX
# note the AWS_MYPEM does not have .pem extension listed
# it found my key that was in ~/.ssh/ and is chmod 600
AWS_MYPEM=XXXXXXXX
AWS_REGION=us-east-1
# Optional if you’re using Amazon’s STS
#AWS_SESSION_TOKEN=””
export AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN AWS_MYPEM AWS_REGION

Source your bash profile to make sure the new variables are active:

jmiller11:fcs2-chef-repo jmiller$ . ~/.bash_profile

Ok lets test that your setup correctly by running “knife ec2 server list”, likely this will be empty for your but as long as it returns the header your fine:

jmiller11:fcs2-chef-repo jmiller$ knife ec2 server list
Instance ID Name Public IP Private IP Flavor Image SSH Key Security Groups IAM Profile State
i-b9ea45e9 base1204-1 m1.small ami-0145d268 aws-jmiller default terminated
i-88e34cd8 m1.small ami-0145d268 aws-jmiller default terminated
i-25e84775 base1204-1 m1.small ami-0145d268 aws-jmiller default terminated
i-c7e44b97 base1204-1 m1.small ami-0145d268 aws-jmiller default terminated
i-41e14e11 base1204-1 54.227.113.203 10.236.185.159 m1.small ami-0145d268 aws-jmiller www, default running
i-7dfc532d base1204-2 54.237.5.212 10.151.112.113 m1.small ami-0145d268 aws-jmiller www, default running
i-53b57800 webserver1 t1.micro ami-3202f25b me default terminated
jmiller11:fcs2-chef-repo jmiller$

Launch Command:

Ok here I am using a simple role called “base” that was uploaded to my chef hosted account all it does at this point is setup chef to run as a cron job to save memory. The ami is a ubuntu 12.04 that will be running on a m1.small instance, with the “default” and “www” security groups, with a easy to read name of “base1204-1” using the ssh key file for my aws key. I need to figure out if there is a better way then defining he ssh key on the line but this works for now.

jmiller11:fcs2-chef-repo jmiller$ knife ec2 server create -r ‘role[BASE]’ -I ami-0145d268 -f m1.small -x ubuntu -G default -N base1204-1 -i ~/.ssh/aws-jmiller

The output of the command will run and you should see something like this:

jmiller11:fcs2-chef-repo jmiller$ knife ec2 server create -r ‘role[BASE]’ -I ami-0145d268 -f m1.small -x ubuntu -G default,www -N base1204-2 -i ~/.ssh/aws-jmiller
Instance ID: i-7dfc532d
Flavor: m1.small
Image: ami-0145d268
Region: us-east-1
Availability Zone: us-east-1a
Security Groups: default, www
Tags: Name: base1204-2
SSH Key: aws-jmiller

Waiting for instance…………………
Public DNS Name: ec2-54-237-5-212.compute-1.amazonaws.com
Public IP Address: 54.237.5.212
Private DNS Name: ip-10-151-112-113.ec2.internal
Private IP Address: 10.151.112.113

Waiting for sshd….done
Connecting to ec2-54-237-5-212.compute-1.amazonaws.com
ec2-54-237-5-212.compute-1.amazonaws.com Installing Chef Client…

ec2-54-237-5-212.compute-1.amazonaws.com Chef Client finished, 7/12 resources updated in 14.133802702 seconds

Instance ID: i-7dfc532d
Flavor: m1.small
Image: ami-0145d268
Region: us-east-1
Availability Zone: us-east-1a
Security Groups: default, www
Security Group Ids: default
Tags: Name: base1204-2
SSH Key: aws-jmiller
Root Device Type: ebs
Root Volume ID: vol-cc24df85
Root Device Name: /dev/sda1
Root Device Delete on Terminate: true
Public DNS Name: ec2-54-237-5-212.compute-1.amazonaws.com
Public IP Address: 54.237.5.212
Private DNS Name: ip-10-151-112-113.ec2.internal
Private IP Address: 10.151.112.113
Environment: _default
Run List: role[BASE]

Ok thats the basics, if you get this far you might want to checkout chef-metal