by w4l3XzY3

by w4l3XzY3

Cloud Computing and Cloud Infrastructure Myths

Share this: [wpsr_retweet] [wpsr_plusone][wpsr_linkedin]

Most common cloud computing questions

The most common question we hear about cloud computing is What is the cloud?. There are a lot of terms, vendor specific definitions and confusion about cloud infrastructure so we’ll first define cloud computing before moving on.

Solid Logic’s cloud computing definition: Instantly scalable, programmatically-controllable compute, storage, and networking resources. 

This definition is also commonly referred to as Infrastructure-As-A-Service (IaaS). Infrastructure-as-a-Service abstracts the physical aspects of IT Infrastructure and provides a set of application programming interfaces (APIs) to control all aspects of the infrastructure. It is very powerful and allows you to basically manage a data center from a development environment or software application.

Many of the people we speak to have never used Amazon Web Services (AWS)Rackspace Cloud or another IaaS cloud provider, for different reasons. We’ve used IaaS for everything from High-Performance Computing to Video Hosting to low-cost development/test or non-production infrastructure. Our experience serves as a guide on which workloads fit well within an IaaS structure and which ones do not. It also allows us to prescribe  a customized, phased approach to cloud integration that minimizes cost and business risk in different ways.

The next comment that normally comes up when speaking to people about cloud infrastructure is: “The cloud sounds great, I hear it saves a lot of money but its just too risky/insecure/complex for us.” 

Organizations that have not yet embraced IaaS or “the cloud”  in their business generally do so for similar reasons. Most of the reasons center around perceptions that may be outdated or untrue – it depends on their scenario.

In our experience, their reasons generally fall into one of the categories below:

  • Cloud Performance (CPU, Disk, Network, Bandwidth, etc.) – I heard cloud servers are slow. The disks are slow and unpredictable. 
  • Budgeting/cost modeling – How do I know or estimate what my costs will be?
  • Cloud Security – It can’t be secure. Its called ‘Public Cloud’. Can other people access my files or servers?
  • Cloud Reliability – Netflix went down so it’s not reliable. What do I do if it goes down?
  • Cloud Compliance – No way, can’t do it – I’m subject to ABC, DEF or XYZ compliance requirements
  • Cloud Audit requirements – No way, the auditors will never buy-in to this.
  • Employee training – How do I find people to manage this?
  • Steep learning curve – How do I get started? Its seems really complex.

Cloud misperceptions abound

As the saying goes, perception is reality, and there are also a lot of misconceptions that increase fear of the technology and prevent people from moving suitable workloads to the cloud.

Popular news sources perpetuate the myths about cloud computing. It seems that every time Amazon Web Services (AWS) (who is by far the largest cloud provider) has any sort of hiccup or downtime, reporters jump on the bandwagon that cloud infrastructure is useless and breaks too often. Here is a link to a Google news search for this: https://www.google.com/news?ncl=dvYSd5T83PVQigMPa1-2GMz-snaDM&q=aws+down&lr=English&hl=en

[wdca_ad id=”2619″ ]

How we’re addressing these concerns

We’re going to address each of these concerns by sharing much of what we’ve learned along the way. We hope to shed some light on what seems to be an increasingly complicated market with more and more terminology and complex jargon used every day.

  1. We’re working on a comprehensive cloud computing benchmarking report. The report will make an apples-to-apples comparison between cloud instance sizes and existing in-house infrastructure. It will use common benchmarking tests that anyone can replicate in their environment. It will allow organizations to make informed business decisions on whether or not they could benefit from integrating “the cloud” into their IT infrastructure and software development approach. Sign up here for a copy of the cloud computing benchmark report.
  2. We’re going to present some cost models and budgets for common scenarios. We’ll integrate both tangible and non-tangible costs and benefits that we’ve searched for but haven’t seen included anywhere else. Contact us for a cost model for a specific use case.
In all we’ll address each of the bullet points above in detail. Stay tuned…

[gravityform id=”4″ name=”Subscribe to our Blog” description=”false” ajax=”false”]

Share this: [wpsr_retweet] [wpsr_plusone][wpsr_linkedin]

Event: AWS Michigan Meetup (Presenting) – 10/09/12

Legal Informatics w/ CloudSearch & High-Performance Financial Market Apps

Solid Logic’s CEO, Eric Detterman, and CIO, Mike Bommarito, will be presenting at the <a title="AWS Michigan Meetup" href="http://www.awsmichigan doxycycline tablets for acne.org/events/85530922/”>AWS Michigan Meetup at Tech Brewery (map) in Ann Arbor, MI. I’ll be presenting on how we use Amazon Web Services (AWS) in the quantitative financial trading space with a case study and more.

Mike will be presenting on Legal Informatics using AWS CloudSearch. He will also be demonstrating an early prototype of an private enterprise information search and e-discovery application we’re creating. Mike also has a copy of his presentation available here.

Event Date: Tuesday, October 9th, 2012 @ 6:30pm

Event: AWS Michigan Meetup

More info: http://www.awsmichigan.org/events/85530922/

Below is a copy of my presentation so you can view at it at your convenience.

[gravityform id=”4″ name=”Subscribe to our Blog” description=”false” ajax=”false”]

Amazon EC2 Cloud Computing Cost Savings

This post is a long one and is part of an on-going series of some of the benefits we’ve identified in our experience in using Cloud Computing technologies, most notably Amazon Web Services (AWS) and different VMware products.

Overview

“The cloud”, specifically Amazon Web Services, has dramatically changed the landscape of High Performance Computing and Big Data Processing in recent years. Many things are computationally possible that would not have been a few short years ago. An organization can cost-effectively setup, launch and use a seemingly limitless amount of computing resources in minutes.

Most of the news media today is focused around using Hadoop on “Big Data”. SLTI has experience with this technology, but what happens if your task data set doesn’t fit nicely into this framework?? The writeup below is how we handled one such challenge.

Business Problem

The problem SLTI was trying to solve fits into the Business Intelligence/Data Mining area in the financial industry. The problem tested different inputs for an algorithm that is the basis for a quantitative equity trading system.
The algorithm had complex mathematical calculations and processing requirements across a large and diverse data set. The problem required testing a wide range of input parameters across four dimensions. The algorithm was tested across sixty-two different data sets. A summary of the size of the problem is shown to the right – We basically have to analyze 9.9 billion data points to come up with something actionable.

While the program logic is specific to the financial trading industry, it has many common concepts shared across different industries – engineering, legal services, etc. The question to ask is simple –

How many processing tasks have you had that you wished ran faster? Can they be split into multiple pieces and run in parallel? How can this be done cost-effectively?

Information Technology and Software Solution

Cloud Computing has dramatically changed the cost landscape of IT Infrastructure, especially for prototype or short run projects like this one. In a general sense, CPU cycles and RAM are cheap compared to the highly skilled labor required to improve performance by several orders of magnitude.

Our goal was simple – make the program to run a lot faster with minimal effort.

We have a large list of projects to be completed so development time is our most precious resource so we didn’t want to re-write the entire program. We kept the software changes and technology solution simple – it’s basically an 80/20 approach to setup the infrastructure and handle the code changes that still solves the problem, albeit in a less elegant fashion.

To accomplish our goal, we modified the program  to operate on a user-defined subset of the original data set. This allows the problem to be split into many small parts and spread apart across multiple servers. We can then distribute the pieces to each server to handle the processing for that subset.  

IT Infrastructure Architecture

In staying with a 80/20 simple solution first approach, we created a solution with the following pieces:

  1. Linux based application server (Amazon EC2 Amazon Machine Image (AMI), alternatively a VMware image could be created and converted to an AMI.
  2. Highly-Available, scalable, central filestore (Amazon S3)
  3. Master configuration data stored in Amazon S3

The cluster itself is comprised of sixteen cc2.8xlarge EC2 instances. Each instance has 88 Compute Units, has 2 x Intel Xeon E5-2670 processors (16 cores per instance), 60.5GB of RAM, 3370GB storage. The cluster provided 1408 Compute Units, 256 Cores and 968 GB of RAM.

The basic logic of the program goes something like this

  1. Load all required data into Amazon S3
  2. Launch the pre-configured AMI to run the program after the server launches
    • Get a specific subset of the data for the node from the central filestore
    • Update the master configuration data to notify the other nodes what data still needs to be processed before, during and after each test run.
    • Save the results to the central filestore
    • Shutdown the node after the work is completed

Cost Analysis

This is not intended to be a totally comprehensive cost comparison but rather a quick TCO comparison using some standard costs. To quickly do this, we used the AWS EC2 Cost Comparison Calculator on the bottom of this page.

SLTI’s EC2 based approach is roughly 99.5% cheaper than an in-house solution.  There are other similar examples of the ROI of an EC2 based approach for this type of workload here 

Key Takeaways

  1. Using the cloud enables a much more adaptive and scalable data processing infrastructure than in-house IT hardware.
  2. If you’re not using AWS (or something similar), you’re overpaying for IT infrastructure, especially for short run or highly variable workloads.

This post is a short overview on some of the ways we’re using advanced cloud computing technology to help our clients improve their IT agility and reduce IT expenses. We’re currently working on a few case studies that describe these concepts in more detail. To get updated with new research just sign up using the form on the right of this page.

If you’d like to explore a specific use case for your situation – please contact us

[gravityform id=”4″ name=”Subscribe to our Blog” description=”false” ajax=”false”]

Cloud Computing Categories

This post is the second in an on-going series of some of the benefits we’ve identified in our experience in using Cloud Computing technologies, most notably Amazon Web Services (AWS) & different VMware products. The first post focused on some of the financial benefits and cash flow impacts of technologies we use on all client projects.  This post will introduce some differences between “cloud computing” technologies in order to set the statge to discuss how the products in those categories can be used and some of the benefits of them.

Cloud Computing Definitions & Categories

Because “the cloud” is a very vague and overused term these days – we need first define some things before diving into the impact and benefits of them. In the interest of time, we’ll use a summary of Wikipedia’s definitions rather than creating our own.

Virtualization – “In computing, virtualization (or virtualisation) is the creation of a virtual (rather than actual) version of something, such as a hardware platform, operating system, storage device, or network resources.” The most notable example of this software is made by VMware (ESX, Workstation, etc.).

Source: http://en.wikipedia.org/wiki/Virtualization

Cloud Computing & Infrastructure As A Service (IaaS) – “Cloud computing refers to the delivery of computing and storage capacity[citation needed] as a service to a heterogeneous community of end-recipients. The name comes from the use of clouds as an abstraction for the complex infrastructure it contains in system diagrams[citation needed]. Cloud computing entrusts services with a user’s data, software and computation over a network.”  Amazon Web Services (AWS), Rackspace, etc doxycycline tablets 50mg. would fall under this category.

Source: http://en.wikipedia.org/wiki/Cloud_computing &

Platform As A Service (PaaS) – “Platform as a service (PaaS) is a category of cloud computing services that provide a computing platform and a solution stack as a service. Along with SaaS and IaaS, it is a service model of cloud computing. In this model, the consumer creates the software using tools and libraries from the provider. The consumer also controls software deployment and configuration settings. The provider provides the networks, servers and storage.” Heroku, PHPFog, AppFog, etc would fit into this category.

Source: http://en.wikipedia.org/wiki/Platform_as_a_service

Cloud Computing Technology Stages

These categories have been introduced over the past nine or so years and have matured considerably in recent years. Many of the products that we looked at a year ago and felt were not ready for prime time are now ready Our use of cloud computing technology has matured considerably and we’ve followed the path shown in the diagram below.

Our preference at this point is to use a Platform-As-A-Service where appropriate. When that is not possible, we’ll use a customized configuration running on Amazon Web Services or another cloud provider.  In the next post we’ll discuss some of the benefits of this approach and the impact these decisions have had on our development process.

[gravityform id=”4″ name=”Subscribe to our Blog” description=”false” ajax=”false”]