Archives for May 2012

Cloud Computing Categories

This post is the second in an on-going series of some of the benefits we’ve identified in our experience in using Cloud Computing technologies, most notably Amazon Web Services (AWS) & different VMware products. The first post focused on some of the financial benefits and cash flow impacts of technologies we use on all client projects.  This post will introduce some differences between “cloud computing” technologies in order to set the statge to discuss how the products in those categories can be used and some of the benefits of them.

Cloud Computing Definitions & Categories

Because “the cloud” is a very vague and overused term these days – we need first define some things before diving into the impact and benefits of them. In the interest of time, we’ll use a summary of Wikipedia’s definitions rather than creating our own.

Virtualization – “In computing, virtualization (or virtualisation) is the creation of a virtual (rather than actual) version of something, such as a hardware platform, operating system, storage device, or network resources.” The most notable example of this software is made by VMware (ESX, Workstation, etc.).


Cloud Computing & Infrastructure As A Service (IaaS) – “Cloud computing refers to the delivery of computing and storage capacity[citation needed] as a service to a heterogeneous community of end-recipients. The name comes from the use of clouds as an abstraction for the complex infrastructure it contains in system diagrams[citation needed]. Cloud computing entrusts services with a user’s data, software and computation over a network.”  Amazon Web Services (AWS), Rackspace, etc doxycycline tablets 50mg. would fall under this category.

Source: &

Platform As A Service (PaaS) – “Platform as a service (PaaS) is a category of cloud computing services that provide a computing platform and a solution stack as a service. Along with SaaS and IaaS, it is a service model of cloud computing. In this model, the consumer creates the software using tools and libraries from the provider. The consumer also controls software deployment and configuration settings. The provider provides the networks, servers and storage.” Heroku, PHPFog, AppFog, etc would fit into this category.


Cloud Computing Technology Stages

These categories have been introduced over the past nine or so years and have matured considerably in recent years. Many of the products that we looked at a year ago and felt were not ready for prime time are now ready Our use of cloud computing technology has matured considerably and we’ve followed the path shown in the diagram below.

Our preference at this point is to use a Platform-As-A-Service where appropriate. When that is not possible, we’ll use a customized configuration running on Amazon Web Services or another cloud provider.  In the next post we’ll discuss some of the benefits of this approach and the impact these decisions have had on our development process.

[gravityform id=”4″ name=”Subscribe to our Blog” description=”false” ajax=”false”]

Michigan Lean Startup Conf. Twitter Visualizations

Today I am out at the Michigan Lean Startup Conference in Grand Rapids, MI. The conference is put on by Momentum…catch us on Twitter @solidlogictech

Our new CIO, Michael Bommarito, created a couple quick visualizations of the activity of the Twitter hashtag for the conference #leanstartupmi. We used a combination of Python and R code to do this. We’ll update the charts after the conference concludes.

Here they are:

[gravityform id=”4″ name=”Subscribe to our Blog” description=”false” ajax=”false”]

When will R overtake SPSS and SAS?


R (formally ‘The R Project for Statistical Computing’) is growing quickly in usage and the user base. Based on some recently published forecasts R will overtake SAS & SPSS around 2015.

SLTI Commentary:

We use R for many different items and just got back from the R in Finance 2012 conference last week. Check out our post about thoughts on it here. The findings about R possibly overtaking more commercial languages are interesting, but its a bit of an apples to oranges comparison – R is viewed, in a general sense, as free vs. SAS & SPSS which are commercial packages.  I would be more interested to see some kind of market share or other metrics to show the change occurring in larger commercial installations. We met some of the people from Revolution Analytics last week and are very impressed with their solution – They should do a lot to increase the use of R within the enterprise.

Any ideas on if or when R will overtake either SAS or SPSS in commercial use?

Source: & http://blog Discover

Cloud Computing Cost Benefits – Cash Flow Impact

We’re heavy users of cloud computing technologies – primarily Amazon Web Services (AWS) and services built on top of it. The decision to embrace this technology has been very important to our client projects. There are many reasons behind this, but here is the first of the major reasons behind the decision (we’ll cover the others in later posts).

Cloud Computing Reduces Costs & Improves the Cash Flow of an Uncertain Project

Many of the projects we work on begin with some kind of prototype to confirm the client’s expectations or project goals. If the project goes well, the client then moves on to a larger roll-out and larger subsequent phases. If it doesn’t then the project ends. Because of this, there is a lot of variability and uncertainty in any estimates used to project infrastructure requirements or pricing models. Our preferred method of managing this risk is to use a variable cost compute model and leading services (IaaS, PaaS, SaaS, etc). While doing this, we develop everything to be standard compliant so we can switch to a better option in the future as the clients needs change.


  • Recently we completed a prototype/proof of concept project with an expected internal user base of around 300 users. About a week after launch we ended up with 750+ internal users due to the popularity of the program. We were able to transparently scale up once there was sufficient demand (not before) with zero downtime. We also did this using a variable-cost model so the ‘estimation risk’ to the client was greatly reduced.
  • We designed the infrastructure for a site that hosted and displayed user submitted videos for a large internet based comedy contest. There was no accurate way to predict how popular the contest would be so we used an entirely variable-cost model to build and deploy the solution. The solution used a few different AWS This allowed the site to scale up and down based on actual demand and traffic, which we believed was a reasonable proxy for revenue.
  • We’re currently working on an advanced prototype project now that does some complex analysis on ~210GB of structured and unstructured data. Again, the project is a prototype with an uncertain future. Instead of taking a leap and estimating what size hardware to buy or making any other decisions based on limited information, we’re using Amazon Web Services to build a cluster of servers to test our product. Our projections show we’ll be able to build the cluster for about 80-90% less than the cost of a single server. This estimate doesn’t factor in other items necessary to make a complete comparison, but is a good starting point.


There are certainly some situations where ‘the cloud’ doesn’t make sense financially or operationally, but generally if the pricing model full factors everything in, and takes into account a measure of uncertainty, the benefit is clearly on the side of cloud services like AWS.We’ll go through an example cost model in the future to further clarify this. Until then, the AWS pricing tool should give some good ideas on pricing comparisons – especially under situations needing lots of computational power for a short time. Its pretty clear that for certain use cases, the cloud is the only way to go from a pricing standpoint.

Its a whole different animal to estimate the requirements and cost model for a well defined and modeled project. If the project you’re working on has a high variability (i.e. standard deviation) then its almost certainly more cost-effective to go with a cloud solution. Even if the costs are equal or slightly higher, we feel its cheap insurance and safety to go with a more adaptive solution.

[gravityform id=”4″ name=”Subscribe to our Blog” description=”false” ajax=”false”]

RinFinance Conference Thoughts

Thursday and Friday of this week our new CIO, Michael Bommarito and I were in Chicago for the R in Finance conference. The conference was great, with a lot of very well done presentations. The R project has come a long way and is growing quickly. We use the R language for some of the more complex analytical projects we work on with great results. While the conference subject matter was very specific to the quantitative finance field, there were many things that carry over into other fields. A couple of the main ones are below:

Business Intelligence Labor Market

The Business Intelligence (BI) labor pool for people with skills in Big Data, Business Intelligence and enterprise level statistics and analytics area looks to be small, relative to other IT fields, and seems to be centered within specific geographic areas (NYC, Chicago, Silicon Valley, etc.). The skilled people are in high demand now and wages are up across the board. If your company is outside of these areas or not a large operation with a large budget for BI your options are fairly limited and generally priced accordingly. This tightness is in part due to the large growth this area is seeing, but also the newness of the field and the speed of innovation currently seen.

Commercial Open Source Software

The R project, being an open source project has a wide spectrum of users. At the conference there were people from large, leading hedge funds and investment firms to students and hobbyists. This wide user base seems to be unique to open source software and even more unique is the fact that it all seems to work so well and support different requirements well. Quick example….a student doesn’t have the money to pay for an enterprise level BI software package so he/she doesn’t use it. He/she may use a free or open source alternative product click resources. The enterprise customer generally won’t use an open source product without dedicated technical support or  community support, unless they have in-house talent to fix issues that may come up. This can leave the software project in a bind because of the disparity between the users and then people that would pay money to sustain development.

One solution to this is commercial open source software. Some easy examples of this are Red Hat Linux and MySQL (now part of Oracle). The commercial company gets a potentially large set of available labor to develop and work on the products and the students have a job market open for them with their new skills. The end client ends up with a additional layers of ‘insurance’ and support – from their chosen consulting company, the commercial company supporting the product, and the original the large open source community of free users – generally at a lower cost than in-house developed or proprietary solutions.

This is likely one of the reasons that we’ve seen large growth in the market share of commercial open source software in recent years.

Image courtesy of

[gravityform id=”4″ name=”Subscribe to our Blog” description=”false” ajax=”false”]