"Personal Compute Cloud" Infrastructure Code

[https://github.com/majidaldo/personal-compute-cloud]

Problem: Automate computing infrastructure setup
Solution: Docker hosts on CoreOS machines provisioned with Ansible.

I've recently finished coding up a solution to tackle 'personal' distributed computing. I was bothered by the (apparent) lack of a framework to handle the coordination of setting up multiple machines. And shell scripts are messy. Once I learned Ansible, I was not bothered! (It will be the only systems automation tool I will be using in the foreseeable future! yah..Ansible is AWESOME!)


Catering to the Scientific Computing Workflow: However, mere automation was not my only concern. I wanted a seamless transition from what I'm working on locally to being able to bring more computing power from remote machines. Unlike (pure) software engineering there isn't a 'development' environment and a 'production' environment. Now there are a handful of codes out there that can help you provision CoreOS clusters, but that does not fit well with the scientific computing workflow.

Status: Most of the functionality that I had planned has been implemented. However, like all codes, it's a work-in-progress. I'll be adding functionality as needed by my priorities.

Try it out.

Use Vagrant FROM Ansible to Automate Hybrid Cloud Infrastructure

[https://github.com/majidaldo/ansible-vagrant]


The Intro

This is NOT about having Vagrant provision with Ansible. This is about having Ansible treat Vagrant as a provider of hosts.

Building on my previous experience with the 'cloud', I still felt like I needed another tool to script and glue the process of getting my infrastructure up. I started out with shell scripts but they quickly got messy as the complexity increased. I knew about all the devops tools out there but I avoided them because I thought they would be too complex themselves for what I wanted to do which is relatively simple. But I bit the bullet on went full-on devops with Ansible.

Ansible is GREAT! I found it suitable for (technically-minded) beginners. However, it still took me a few days to get the hang of it. I had to get a little bit under the hood since it did not do what I wanted it to do out of the box.

I want to setup something like a hybrid cloud where I run some services locally and just bring up high-performance compute nodes on demand and have them talk with my local services. I use Vagrant to setup local virtual machines. Vagrant is great for development environments but when I want to manage and orchestrate several VMs locally (let alone on the cloud), things can get messy.

So, I (further) developed ansible-vagrant to interface with Vagrant from Ansible (solving cygwin problems along the way).


The Cream

You can, from Ansible

• Set state=(up|halt) for some VM
• Get a Vagrant host inventory
• Get a SSH config for a host
• Destroy VMs