Chef and Knife on Joyent Public Cloud

Are you interested in Chef and want to give it a spin on Joyent Public Cloud? This article will go step by step from creating an account with Joyent to listing, creating and deleting servers using Chef's Knife tool.

Getting started with Chef Server on Joyent Public Cloud is so easy it's almost embarrassing.

Pre-Requisites

The following are expected knowledge and assets. If you do not have these, please learn them or configure them before proceeding.

  • A working knowledge of SSH
  • Familiarity with SSH public keys
  • An SSH public key

Procedure

Configure a Joyent Public Cloud Account

  1. Visit https://my.joyentcloud.com/ and click the Create an account button (follow the procedure and log in)
  2. If you didn't add your SSH key during account creation, click on your your.email@example.com in the top left and click SSH Keys and add your key

Create a chefserver SmartMachine

  1. Log into your account add click the Add a machine button
  2. Under Images | Select an Image search for chef and click on the chefserver box that appears
  3. Under Sizes select Medium 2GB (an Extra Small won't provision successfully at the moment)
  4. Under Confirm & Customize click Provision
  5. Wait for it to show as Running

Configure a Chef Client (knife.rb and username.pem)

  1. Log into the public IP of your chefserver SmartMachine as root using SSH (having your public key added to your SSH agent makes this easier)
  2. Create a Chef Client with knife configure --initial and accept defaults for everything except chef server URL for which you should enter http://<public-ip>:4000 and clientname for the new client for which you should enter your_name

Configure a Chef Workstation

In the interest of ease of setup for a proof of concept, I recommend configuring this on the Chef Server:

  1. Install basic Ruby and build system packages (ex: pkgin -y install build-essential libxslt ruby193-rubygems ruby193-readline)
  2. Create an SSH Key on the workstation with a strong password (ex: ssh-keygen -b 2048 -t rsa)
  3. Back on the Joyent Cloud web UI: Add the content of /root/.ssh/id_rsa.pub to SSH Keys and record the name you give it
  4. Install the Ruby Gems that support provisioning  (ex: gem install chef net-ssh net-ssh-multi fog highline knife-joyent --no-ri --no-rdoc)
  5. Get a blank Chef repository to work from (ex: git clone https://github.com/opscode/chef-repo.git)
  6. Under chef-repo create a .chef directory
  7. From your Chef Server copy /root/.chef/your_name.pem and /root/.chef/knife.rb to chef-repo/.chef/
  8. Work around the fact that Chef install.sh doesn't support SmartMachines:
  9. Add lines to chef-repo/.chef/knife.rb for Joyent provisioning with Knife:

    knife[:joyent_username] = 'your.username@example.com'
    knife[:joyent_keyname] = 'your_keyname' # look under SSH Keys in the Joyent Cloud web UI
    knife[:joyent_keyfile] = '/root/.ssh/id_rsa'
    knife[:joyent_api_url] = 'https://api.joyentcloud.com'

Verify that Knife Works

  1. Try to list the Machines you have with knife joyent server list.  When prompted for PEM pass phrase, enter it and see that your chefserver SmartMachine is listed:

    [root@b5c03525-beb0-43b0-ae37-c5b69568c8a7 ~/chef-repo]# knife joyent server list
    Enter PEM pass phrase:
    ID Name State Type Image IPs RAM Disk Tags
    b5c03525-beb0-43b0-ae37-c5b69568c8a7 jpc-mychef2 running smartmachine sdc:sdc:chefserver:1.1.0 10.112.6.253 165.225.139.87 2 GB 60 GB
  2. Try to create a new SmartMachine with knife joyent server create -N "demo.example.net" --flavor "Extra Small 512 MB" -I 60ed3a3e-92c7-11e2-ba4a-9b6d5feaa0c4 -d joyent-smartmachine.
    • You should see demo.example.net listed under Nodes on http://<chefserver-ip>:4040 (log in as chef with the password from Credentials on the Machine page in the Joyent Cloud web UI)
  3. Next, we'll try to delete a Machine.  List your Machines with knife joyent server list and record the ID of one that IS NOT your chefserver.  That would be awkward ;)  Try to delete that non-chefserver SmartMachine.with knife joyent server delete <ID>.

If all that works, you are set to go.  At this point you are free to roam the universe of Instrastructure as Code with a Joyent's high performance playground to prove it on.

Caveats with Provisioning SmartMachines

This all works seamlessly for custom Chef Recipes and for Linux VMs on Joyent Public Cloud.  However, there are some hurdles to be aware of if you are provisioning SmartMachines with Chef on Joyent Public Cloud:

  1. Default Chef Bootstrap using install.sh will fail (hence the custom boostrap I explain in the steps above)
  2. Package names sometimes differ from Debian/Redhat
  3. File system layout is different from Debian/Redhat
  4. Most of Opscode's Chef Recipes don't account for the above (Opscode are actively working to improve this)

There is light at the end of this tunnel.  Opscode, Joyent, the SmartOS and Joyent Public Cloud communities are all working to close the gap on Chef support for SmartMachines.  In the meantime, take a look at ModCloth's Cookbook on Github to potentially use them or simply learn some of ModCloth's techniques for writing Chef Recipes for SmartMachines.

Conclusion

It is straight-forward to get started with Chef on Joyent Public Cloud.  Once you get past the initial hurdles having an architecture that supports multiple clouds and operating systems will give you, your team, and your company the flexibility to choose the best price/performance cloud and platform.  As of this writing those two are Joyent Public Cloud and SmartOS.