Thursday, June 15, 2017

Skinny dipping in the Cloud

On a recent trip to the beach with my family, I noticed something very interesting. Not everyone was wearing the same swim suit. I saw every size, shape and color: itsy bitsy bikinis, wetsuits, boardshorts, and sun suits.

Every time you jump into a pool or ocean you have several decisions you need to make, but the most
important decision is what to wear. If you live in Northern California, like I do, you would be crazy to get in the water without a wetsuit. It is just too cold. But on the beaches in Hawaii in the summer wearing a wetsuit you would over heat not to mention be laughed off the beach from the locals. Then can you imagine what would happen if you showed up to a nudest beach in a wetsuit. You might even get arrested. :)

It is important to pick the right apparel for the right location, weather and occasion.  Not doing so could be embarrassing or land you jail. The same might be true of building out infrastructure in your Data Center. Many technologies claim theirs is the best and that is all you need. I would say it completely depends. It is naive  to think that containers or VMs should be the only technology in your data center. Just like the person that shows up to a nudist beach in a wetsuit.

So why are Private and Public Cloud vendors pushing Virtualization or Containers everywhere? There isn't one technology that solves all of your problems. It is important to understand the benefits of each of the technologies and how best to use them. With a large number of technologies I will  focus on compute technologies: Virtual Machines(VMs), Containers, and Bare Metal.

Virtual Machines - VMs (Wetsuits) 

Virtual Machines have been around since the 2000s. The largest vendor being VMWare. The technology has a rich ecosystem that has been established over the last 20-30 years. Most organizations have years of experience using VMWare in parts of their data center infrastructure. Here are some of the benefits of using VMs in your data center:

  • Isolation - The complete stack for the application is installed on the virtual machine. OS, Bin/Libs/Data, and applications. 
  • Compaction - Multiple VMs can run on the same machine at the same time, increasing utilization and efficiency in the data center.
  • Portability - Ability to move VMs from machine to another even on different underlying hardware architectures.
  • Migration - Ability to move running VMs from machine to machine to have fault tolerances and disaster recovery.
  • Management Infrastructure - There is a rich ecosystem that has been built over decades to manage, monitor and secure VMs across data centers.
Basically VMs give you protection and the ability to move applications quickly. Just like a wetsuit that protects against cold and rocks and coral at the beach, VMs can be flexible, give you security, isolation and protection.

Containers (Speedo)

Container technology has been around for about the same time as VMs. But they have really not taken off until the 2013 when Docker made using container easier to use. Docker made containers easier for developers to "spin up" new containers to service applications. Not only did they make them easier to use, but they made them "spin up" faster and consume fewer resources. The key behind the technology is sharing an operating system, at the same time giving some sort of process group isolation.
Some of the benefits of Containers are:
  • Process Isolation - Containers are created by creating process groups in the Operating System. Isolating process spaces from each other.
  • Compaction - 100s of containers can run on a single machine depending on the size of the application. Because they share the OS they don't care about the OS overhead.
  • Application Deployment - Docker has made it easy to deploy applications in containers. It has made it easy to repeatably deploy applications over and over again.
  • Spin up speed - Containers can spin up new instances in a matter of seconds, in some cases milliseconds.

The two most visible benefits of containers is speed of spin up, and the small memory footprint for each container giving the ability to put more compute instances on the same machine. In many cases 100s of containers can be put on a single machine where 10s VMs can be put on a machine. Container are just like a speedo. If you are in a swim meet you want the least amount of drag and you want to be fast. So you pick a speedo to where not a pair of board shorts. Containers are great when you need quick spin up and a smaller memory footprint.

Bare Metal (Skinny Dipping) 

Bare Metal means a computer without an intervening abstraction layer. Basically you are running your application on the hardware. Many times the Operating System is chosen based on the type benefits of the operating system and the application working together for optimizations for the application. Latency or speed sensitivity applications typically run on Bare Metal.
The benefits of Bare metal applications are:
  • Optimization  - Applications can be optimized to run on specific Si features (CPUs, or Chip Sets).  The same can be true for the Operating Systems.
  • No overhead - Since there is no abstraction layer or virtualization layer, there is no overhead layer using resources that are not being used for the application.
  • Custom Hardware - Ability to utilize custom hardware with the application is much easier than with containers and VMs.
  • Control - Because you are dealing with the hardware specifically you have complete control of the hardware and its components.
So with bare metal there is nothing between your application and the hardware. Much like skinny dipping in the ocean. You have total freedom. But you may lack protection from sun, sand and surf. These are things you have to take care of yourself. It is not built into the system.

Plan ahead

So why is it that when we talk to Private Cloud Vendors they are trying to sell us a wetsuit (Virtualization) for all of our computing needs. It is because that is how they grew to the size they are today. Virtualization and Containerization gave Cloud vendors the ability to overprovision their machines and get the most $$ for each machine. It is important for you to understand the benefits of each of the technologies and be smart about how you use them. 

In your datacenter you will see a mixture of these technologies. So you want to find a tool that helps you manage VMs, Containers and Bare Metal at the same time. We are starting to see many of the "Cloud Vendors" (OpenStack, VMWare, AWS, IBM Cloud, GCE) to offer management tools for all three. These tools are still in their infancy stage but it is a most welcome change in the industry.


Saturday, May 27, 2017

Am I a Foodie?

Wikipedia definition of Foodie
A foodie is a person who has an ardent or refined interest in food and alcoholic beverages. A foodie seeks new food experiences as a hobby rather than simply eating out of convenience or hunger. 
Google Dictionary Foodie
a person with a particular interest in food; a gourmet.
synonyms: gourmet, epicure, gastronome, gourmand 
During lunch last week someone at work asked me if I had any food aversions. I said I would pretty much eat anything. Then I was asked if I was a foodie. I did not know what to say. I have never considered myself a foodie. Mostly because when I think of foodie I think of men in black turtlenecks drinking wine and eating expensive small plated meals. The Google definition of Foodie is definitely not me.  But Wikipedia describes me perfectly.

I have eaten $1.50 Van De Camp Fish sticks and a meal at a Michelin Star restaurant.

So you decide. These are some of the great meals I have had over the last month.
Van de Camp Fish Sticks with Ketchup @ Home
Brings back memories of being a kid.

Sashimi @ Kinoshita Restaurant in Sao Paulo Brazil

Wagyu Beef @ Kinoshita Restaurant in Sao Paulo, Brazil

Lamb Osso Bucco @ Josephine's in Sao Paulo Brazil

Gnocchi Ragu @ Josphine's in Sao Paulo Brazil

Tempura Soft boiled egg in Tokyo, Japan

Raman Noodles with Boiled Egg and Pork in Tokyo Japan

Traditional Japanese Lunch in Tokyo, Japan.

BBQ Beef, Poutine and Fries @ Montreal Airport, Canada

Cheese Potatoes and beef Foil Dinner
@ 11 year old scout campout

Monday, May 15, 2017

Tokyo 1st Ward - An Expat story

Another trip abroad and another visit to church on Sunday. I am so grateful that I have the Church in my life. I enjoy attending church when I travel; it gives me a feeling of belonging no matter where I am. This time I am in Tokyo, Japan. I got onto and found the nearest church building to my hotel. It was Tokyo 1st Ward, an English speaking expat Ward. Expat Wards are very different from the typical Ward because of their transient nature. Most Expats are in country for 2-3 years and then go back home to the United States or their home country. That means that there is a 1/3 turn over every year in the Ward. Imagine being the Bishop in a Ward like that when all of your volunteers for teachers, youth programs and music are changing every year. Must be hard to have any consistency in the Ward.

I sat next to a family and started talking about their experience in Tokyo. This time a young family with four little kids under the age of 10. They love Tokyo, so much so that they want to extend their stay from 2 years to 3 years. They said the city is clean, very safe and the people are very nice and friendly. Especially helping them figure out the basic day to day things in life, like grocery shopping.  One of the biggest changes has more to do with Suburban American living compared to Urban living in Japan. "Driving a Car".

As a west coast American, driving is like breathing. We have to do it to survive. We love our cars in California's central valley. And not having a car would be very difficult. I even have a hard time when I travel and don't rent a car. No I did not rent a car in Tokyo. :( I found out from this family that they felt the same way and originally got a car when they got here to Tokyo. But quickly returned it after a couple of weeks. A couple of things caused them to stop driving and start riding bikes and taking the trains.

First, they drive on the left hand side of the road. Thanks to the British, the Japanese drive on the "wrong" side of the road. Second, Tokyo is an ancient city and was designed to confuse and disorient invading armies. So the streets are more like a maze than a grid of streets like we have in the western United States. The labyrinth of one way single lane roads is hard to navigate. Tokyo's subway system is extensive and easy to access any part of the city. Therefore, taking the subway and riding bikes is the way that this family has decided to go.  Can you image going to the local grocery store on your electric bike with two kids in tow? That is what happens in Tokyo. :)

Overall, another great visit to church. And I got a bonus. The church I went to was right next to the Tokyo Temple so I got to see the small but impressive Temple Grounds.

I hope to calm down the travelling the next couple of months. I don't want my home ward (or my wife for that matter) to think I have disappeared.


Friday, May 12, 2017

Bouquet Binaries

I recently dusted off an old project of mine called Bouquet. It basically helps me quickly
architect and design systems. See my original blog here. This blog focuses on the Command Line Interface (CLI) aspect of the Bouquet Project.  The project is hosted on GitHub here

Binary setup

There are several different kinds of binary files that are used in the bouquet pattern.
  1. Top Level Command script - "projectName"
  2. Actor Command Script - "projectName-actorName"
  3. Subsystem Command Script - "projectName-subsystemName"
  4. Command Script - "projectName-actorName-command", "projectName-subsystemName-command", or "projectName-command"
The goal here is that we have a consistent command line interface. For example in the project named caade the following are some commands

# caade init // High level scenario
# caade stack up // Subsystem Command
# caade dev ps // Actor Command

Top Level Command Script

There should be one system command that contains all of the commands for the system using the commander package.
  • The name of the file should be "projectName" in the bin directory.
  • The for each actor there should be a command for the actor. This will give a command line interface for each actor
  • There should be a command for each subsystem as well. This will give the ability for each subsystem to have a CLI.
  • There should be a command for each of the top level scenarios for the system. The following is an example of this top level command file
In this case "caade"
#!/usr/bin/env node

var program = require('commander');

  // Actors
  .command('app <command> <applicationName>', 'Work with applications')
  .command('stack <command> <stackName>', 'Work with applications')
  .command('adm <command> <stackName>', 'Work with applications')
  // SubSystems 
  .command('policy <command> <policyName>', 'Work with Policies')
  .command('cloud <command> <cloudName>', 'Work with Clouds')
  .command('environment <command> <EnvironmentName>', 'Work with applications')
  .command('service <command> <EnvironmentName>', 'Work with servioes')
  .command('user <command> <UserName>', 'Work with Users')
  // Scenarios
  .command('init', 'initalize Caade on your machine')
  .command('up [service-name]', 'Launch an application in a specific environment')
  .command('update [service-name]', 'Update web service with new code')
  .command('run <command>', 'Run a command in specified environment')
  .command('ps <command>', 'List processes for the application')
  .command('kill <serviceName>', 'Kill specific service for the application')
  .command('logs [serviceName]', 'Get logs of the application')
  .command('deploy', 'Deploy an application')

Actor Command Script

This is very much like the Top level command script but limits the commands to the actor The file is named "projectName-actorName" a simple example follows.
In this case "caade-app"
#!/usr/bin/env node

var program = require('commander');

  .command('create <application name>', 'Create an application')
  .command('get <application name>', 'Create an application')
  .command('ls', 'List my applications')
  .command('remove <application name>', 'Remove my application')
  .command('show <application name>', 'show details about my application')
The Controller for this might look something like this AppController.js
module.exports = {

  create: function (req, res) {
    var name = "";  // Default
    var stackName = "";  // Default
    if ( {
      name =;
    else {
      // Return Error "No Application Name specified"
      return res.json({error: "No Application Name specified!"})
    if (req.query.stack) {
      stackName = req.query.stack;
    else {
      // Return error with "No Application Stack specified"
      return res.json({error: "No Application Stack specified!"})
    return Application.find({name: name})
      .then(function (app) {
        res.json({application: app});
  get: function (req, res) { ... },
  delete: function (req, res) { ... },
  list: function (req, res) { ... },
  show: function (req, res) { ... },
  ps: function (req, res) { ... },
  up: function (req, res) { ... },
  kill: function (req, res) { ... }

Subsytem Command Script

This is very much like the Top level command script but limits the commands to the subsystem The file is named "projectName-subsystemName" a simple example follows.
In this case "caade-cloud"
#!/usr/bin/env node

var program = require('commander');

  .command('create <cloudName>', 'Attach a Cloud')
  .command('ls', 'List the Clouds attached')
  .command('remove <cloudName>', 'Remove a Cloud')
  .command('show <cloudName>', 'Show details about a Cloud')

Command Script

Command scripts are where everything really happens. The previous scripts just setup for accessing the command scripts. The naming convention of the command scripts follows the actor and subsystem nomenclature "projectName-actorName-command", "projectName-subsystemName-command", or "projectName-command". The trick of the command is to connect to the rest interface of the system. This should coorespond to the controller with a simalar name. For example if you have actor command script then there should be a cooresponding controller for the actor. This way the REST and CLI APIs are consistent.
The following is an example of a simple Command Script that accesses the rest interface. In this case it shows information about a stack in the system
#!/usr/bin/env node

var program = require('commander');
var Client = require('node-rest-client').Client; // Needed to access the REST Interfacce.
var config = require('./system-config'); // Contains the URL to connect to for the REST Interface
var _ = require('lodash');

var client = new Client();

  .option('-v, --version <versionNumber>', 'Show an application stack with version')

var name = program.args;

// Create the REST Command
var url = config.caadeUrl + "/stack/show?";

if(name) {
  url += "name=" + name[0];

if (program.version) {
  url += "&version=" + program.version;
// Call the REST Interface via HTTP Client.
client.get(url, function (data, response) {
  // parsed response body as js object
  if(data.error) {
  else {
    console.log("Name:" + + "\tVersion: " + data.stack.version);
Another thing that I found useful was having the ability to include the ability to allow the user to add a file as an argument to the CLI. This is good for passing in yaml or json files that can be passed into the Controller. In this case I am passing in a yaml file.
#!/usr/bin/env node

var program = require('commander');
var Client = require('node-rest-client').Client; // Access the REST interface
var config = require('./caade-config');
var YAML = require('yamljs'); // Parse a YAML file

var client = new Client();

  .option('-f, --filename <filename>', 'Create an application stack from file')
  .option('-e, --env <environmentName>', 'Create an application stack for the environment')

var name = program.args;

var url = config.caadeUrl + "/stack/create";
// Taking a YAMLfile and converting to JSON and then passing it into the REST interface.
var args = { headers: {"Content-Type": "application/json"}, data: {} }

if(name) { = name[0];

var definition = {};
// Load the YAML file from the local drive and convert it to JSON.
if (program.filename) { = YAML.load(program.filename);

if (program.env) { = program.env;
}, args, function (data, response) {
  // parsed response body as js object
  if(data.error) {
  else {
    console.log("Stack " + + " has been created for environment " + program.env);
Look for more information on my blog. DWP

Wednesday, May 3, 2017

A Friend, a Parent and Mission Presidents

Serving together forms bonds between people that draw them closer together that last for years. These bonds are typically found in fellow soldiers.  I have not served in war or even in the military; So I cannot understand or even fathom the bond that these soldiers feel one with another. But this last week I got a small glimpse of that feeling form the bonds I have created in serving with others in my church.

A Friend

I had the opportunity to serve with Loren Dalton, (Now President Dalton) in Tracy, California over 20 years ago. There I served as an Elder's Quorum President for the newly called Bishop Dalton. 
A Mormon Bishop is a volunteer position in the Church that is in charge of a congregation of about 300 church members. They are responsible for the spiritual and physical needs of the congregation. An Elder's quorum president works with the Bishop as well as others and is focused on families whose parents are typically between the age of 18 - 45 to 50 years old. 
So back to my story. A worked with Bishop Dalton for about 2 years and enjoyed the friendship that we built working together. Recently I was traveling in Sao Paulo Brazil and found out that Bishop Dalton had been called to be a Mission President in Sao Paulo Interlagos Mission. 
Mission Presidents are asked to serve for 3 years. During the 3 years they take their families and leave behind work and their livelihoods and live in the mission. Mission Presidents are in charge of about 200 missionaries in a geographical area.
It was great meeting with President Dalton and his wife, and after 20 years we had a lot of catching up to do. We talked about kids, families, careers and the ups and downs of life. I was most excited to hear what mission life was like and we talked about how their mission has changed their lives and strengthened their testimonies of Christ and his work. It was great to spend an evening with them. I was impressed at how much the Dalton's cared for their missionaries in the mission. They helped their missionaries through tough times at home, person trials and triumphs during their missions. 

This was somewhat self serving for me because my son Jacob was called to serve in that same mission starting in June. The Dalton's will be leaving before Jacob actually arrives in the mission :(.

In part of our conversation President Dalton asked me who my mission president was 28 years ago when I served in Brazil. I mentioned it was President Paulo Grahl. To my surprise he knew President Grahl and told me he was serving as the President of the Mission Training Center in Sao Paulo. He told me I had to go see him.

A Parent

It might be strange for people that have not served a mission to understand the strong bond that missionaries have with their mission presidents. As a missionary you write a weekly letter to the president of the mission telling them how things are going. Some weeks the letters are just complaining about the work other weeks sharing great experiences. This is your direct line to the president. Additional about every 2-3 months you get to have a 1 on 1 interview with the president to talk about things going on in the mission. The President and his wife quickly become the your leaders/parents for those two years. It is a special bond. One that, as I would soon find out, last a life time.

I took President Dalton's advise and went to the MTC to visit President and Sister Grahl. My missionary parents from almost 30 years ago. When I arrived at the MTC I did not realize, something I would find out later, that this was the day that new missionaries arrived to start their 6 week training in Sao Paulo. So President Grahl was busy working with these new missionaries. I patiently waited for outside of the gates of the MTC to see if they would let me in. About 20 minutes later, a slightly grayer, slightly heavier, man walked out of the MTC. We recognized each other immediately and almost like a scene from a movie we ran up and gave each other a big hug.

I was quickly transported to 28 years earlier when this great man hugged me for the first time, a lost and confused missionary in the Brasilia Brasil airport after a 15 hour trip from California to Brazil to start my mission. All of the feelings of two years of working with this great man came back. The shared experiences, the good and the bad. His great big bear hug is what transported my so quickly back. It was a strong loving hug that a parent gives to a child after years of being away.

I felt like I needed to give him a report about my life, my family, my career, my service in the church. I wanted him to be proud of me. The good kind of proud. We got to talk briefly about our mission time together and I got a great tour of the MTC. Again self serving because Jacob would be living in the MTC in Sao Paulo for about 6 weeks while he is learning Portuguese. I mentioned this to President Grahl and he was excited to have one of his "Filhos" have a child server under him. The first from the United States that would be serving in Sao Paulo.

Next President Grahl surprised Sister Grahl who at the time was talking to the new missionaries about what she always talked to us about. Cleanliness and Obedience. Brings back memories of several missionary conferences where she taught us the importance of having clean apartments and keeping the mission rules.  She took about 30 seconds and remembered my name. I was floored. I could not imagine that someone would remember me so well even after white hair and about 60 lbs. What a great couple of hours I got to spend with my Mission Mom and Dad.

Mission Presidents

Many people don't really understand the value that a mission president has on the lives of the Elders and Sisters that server under them. This last week I got a small glimpse into what Mission Presidents do and the impact they have had on my life. I cannot wait for my son Jacob to form those kind of lasting bonds with his mission president in the next couple of years.


Resurrecting Bouquet


In the 1990s I started dabbling with a new kind of system analysis. Object Oriented System Analysis. I quickly became familiar with all of the great OOA/D tools. The one that stood out for me was Rational Rose. I dove right in and over time became quiet proficient in using the tool. I quickly started writing scripts to to make my life easier and automate repeated tasks. This was the birth of a project named Bouquet.
Move forward 20 years. I am still using UML to design and architect systems, but I also use rapid prototyping technologies like sails, rails and grails. Most recently I am focusing on NodeJS/SailsJS development. I dusted off my old Bouquet specs and started working on resurrecting Bouquet with the latest technologies.
These are the technologies that I am leveraging this time.
  • PlantUML - Textual way of describing UML diagrams
  • SailsJS - MVC framework for NodeJS development
  • Commander - npm module for command line programming for NodeJS
  • GitHub MD - Markdown language for projects in GitHub.
The tools by themselves are very useful. Bringing all the tools together is where I found the most benefit.


PlantUML is a component that lets you quickly write several UML diagrams using Text instead of a drawing tool. It is great for many but not all of the UML diagrams. I have found that it covers everything that I typically need to do for Architectures of systems. UseCase, Component, Class, Deployment, Scenarios, and Activity Diagrams.
One of the benefits of using PlantUML that the text files that your create (*.puml) can be checked in to GitHub. You can also generate image files (png) from the text files (puml) and check in the image files as well. I do this so my design documents in GitHub (Markdown language is used) can reference the images that have been generated. Generating the image (png) files is as easy as typing in a command line.
# java -f design/plantuml.jar myDiagram.puml
Because I am using NodeJS. I can use a npm script command to actually generate all of the my images. Basically I put another target in my package.json file in the root directory that searches all of my design directories and generates the png files.
  "scripts": {
  "design": "java -jar design/plantuml.jar design/*.puml design/**/*.puml",
Now you can generate png files for all of your design diagrams, just type.
# npm run-script design 
To find out more about PlantUML click here You can download the latest jar file for quick image generation here. There is also a Plugin for PlantUML for Intellij and several other IDEs.


SailsJS is a MVC convention over configuration framework for NodeJS applications. This uses a common pattern that can be found in several programming languages today. Examples include Ruby o Rails, and Groovy on Grails.


Commander is a nodejs module for command-line processing. I use this to develop command line interfaces for the systems that I architect. This gives me a quick and dirty way of providing a command line interface with very little lifting.

GitHub MD

MD - Markdown language is used to quickly and easily document a git hub repository. The language allows for simple text based documentation to make it quick and easy.


Using the concept of convention over configurability of SailsJS, I extended the same concepts that already exist in SailsJS and created a design and bin directory in the project root directory. This gives me a place to put the design of the architecture as well as the CLI (Command Line interface) of the system being architected. This is important because most of the architectures I am working have a Web, REST and CLI .

Directory Hierarchy

After a SailsJS project is created a standard directory hierarchy contains several directories and files. I added two additional directories to the top level (bin, design, and test). Next, I add corresponding subdirectories in the design directory as shown below.
  • api - Standard SailsJS directory
  • assets - Standard SailsJS Directory
  • bin - Contains commander binaries
  • config - Stanard SailsJS Directory
  • design - Contains Architecture and Design of the system
    • Actors - Actors of the system
      • - ReadMe for all of the Actors
      • < Actor Name > - Directory for each Actor of the system
    • UseCases - Use Cases of the system
      • - ReadMe file for all of the UseCases
      • UseCases.puml - PlantUML file for all of the Use Cases and Actors
      • < UseCase Name > - Directory for each Use Case of the system
    • Systems - System Components
      • - ReadMe for all of the sub-systems
      • < Sub System Name > - Directory for each sub system.
    • - Top ReadMe for the Architecture and Design
    • Architecture.puml - Top level architecture plantUML diagram
    • plantuml.####.jar - plantUML jar file used to generate png files.
  • tasks - Standard SailsJS Directory
  • test - Contains test for the system.
    • bin - Test the CLI
    • Actors - Test the Actor interactions One Test Suite per Actor with each use case
    • UseCases - Test the Scenarios as described. One Test Suite per Scenario with tests for each different path through the scenario
    • System - Test of each subsystem. One Test Suite for each SubSystem, a test for each of the interface calls.
  • views - Stand SailsJS Directory


I know as I start using this I will add more generated artifacts to the system. So if you have any ideas please let me know. You can find more at the github project

Additional Blogs in a series of Bouquet Blogs