The disruptive forces converging Enterprise Computing, HPC & Big Data

Enterprise Computing is the driving force behind many of the major Cloud Provider’s strategy today. Sure, it was small businesses and startups that drove the early innovation on Amazon Web Services but the Enterprise category is responsible for audacious predictions by the likes of Gartner, forecasting $107billion in Public Cloud spending by 2016. Enterprise Computing is roughly an $800billion business so it’s not surprising that we’re seeing this sort of momentum given the industry is migrating to the cloud on-masse. HPC on the other hand is a relatively niche market today. The overall spend is somewhere around $34billion. Cloud adoption has been modest to-date but this is rapidly accelerating. Big Data, a term almost as overloaded and ambiguous as Cloud, is the fastest growing IT sector right now. At least in hyperbole if nothing else. Analyst CAGR numbers are typically around 40% which is pretty remarkable.

There are two disruptive forces at play that will ultimately see the convergence of HPC, Big Data and Enterprise computing.

Big Data
The hype surrounding Big Data has a lot to do with the value this offers to the Enterprise. As the quantity and complexity of unstructured data increases, so does the need to process it. Businesses are finding clever and innovative ways of turning this data into a source of revenue, not directly, but through a deeper understanding of their business, their customers and the habits exhibited by them.
Typically large silos of structured and unstructured data are spread across various divisions and applications within the enterprise. Big Data technology allows questions of this data to be more easily asked to gain insight that allows companies to serve their customers better and compete more effectively.
So Big Data is about performing analysis and computations against large amounts of data. What about HPC? Yep, same thing. Sure, there are some important distinctions and subtleties that I won’t delve into here but the point is we’re already seeing a strong convergence of Big Data and HPC technology.

Cost and Accessibility

Putting Big Data hype aside for a moment, the second, and most important, disruptive force at play is Cloud Computing. The cloud offers many advantages over traditional on premise computing that make HPC-like capabilities far more accessible and affordable. Here is a summary of some of the benefits;

Cost: For spikey workloads, the cost savings in the cloud can be significant. What’s more, the cost is an Operational rather than Capital Expense so is often more palatable for many businesses as costs can be attributed to a particular project.
Ease of use: The cloud can facilitate easy dynamic provisioning for specific workloads. The ability to have OS/Software configurations particular to a workload is a key benefit.
Speed of deployment: The ability to rapidly provision new environments/clusters in minutes is incredibly empowering to many businesses.

Scalability: Elastically scaling out to meet increased capacity demands is a powerful concept. The public cloud promises “infinite” scale. The reality is somewhat different: there are some real limits even in the cloud. But the computing capacity that can be obtained from large providers such as Microsoft Azure is far greater than what most users can muster with internal hardware.
Resiliency: The ability to snapshot workloads as they are running can allow for check-pointing of MPI workloads. Combine this with active monitoring and the ability to dynamically move a guest VM from one physical host to another, and your workloads can keep running even in the face of hardware failure.

Portability: The ability to move a workload from one cloud platform to another on the fly without any application changes allows for powerful options such as for bursting from a private cloud out to a public cloud, High Availability where a workload is run on multiple clouds simultaneously, scaling across multiple clouds to meet extremely high resource requirements or, to take advantage of shifts in the spot pricing market.

In summary, enterprises across many industries have much to gain in adopting Big Compute and Big Data technology and taking advantage of the many benefits Cloud Computing offers for these applications. This is why I predict wider adoption and convergence of these technologies is inevitable. Software Vendors and Cloud Service Providers that are focusing solutions to this market and address challenges around data management and governance will be well positioned.

Follow me on twitter:@dave_fellows

Posted in Cloud, HPC, Microsoft, Security | Tagged , , , , | Leave a comment

GreenButton’s Premiere Google Hangout Event

On Friday afternoon, we had our premiere Google Hangout – #GBHPTalk – and it was amazing. Attending on behalf of GreenButton was our CTO Dave Fellows and our CEO Scott Houston. Joining Dave and Scott were two well-known cloud evangelists, Ben Kepes and HP’s Stephen Spector.

A variety of topics were discussed such as: What do businesses really need to think about when looking to migrate to the cloud? What opportunities does it facilitate – is it just costs savings, operational efficiencies or something more compelling like being able to compete more effectively? Why security matters and what do organizations need to know when considering a cloud vendor? Who’s doing it right and what risks do open source vendors pose? And, the closing conversation focused on trends – from inside GreenButton and HP on the matters of governance and data management.

If you were not able to attend #GBHPTalk, we’d like to invite you to watch the video replay and hear the interesting dialogue.

Tell us what you think about the topics and feel free to comment.

We’ll be doing another Google Hangout next month, so stay tuned for the date and the topic.

GreenButton's Google Hangout

Posted in GreenButton | Leave a comment

Burst your Application to HP Cloud with the GreenButton Cloud Fabric

It’s our privilege to have Stephen Spector, Social Media Strategist with HP Cloud, contribute to our blog. Stephen has many years experience as an open source community manager for OpenStack and Xen.org while at Rackspace and Citrix Systems. Prior to managing Xen.org at Citrix Systems, Stephen founded the Citrix Developer Network, developed global alliance and licensing programs, and even once wrote software that may be still running on your computer…

The advancements in Big Data and Cloud Computing continue to drive enterprise customers to maximize their existing resources while also taking advantage of the public cloud advantages for bursting. GreenButton and HP Cloud have joined together to deliver the industry’s leading open source cloud bursting capability running on the state of the art OpenStack cloud platform.

Dave Fellows, CTO at GreenButton and I recently recorded a Podcast to discuss this unique enterprise opportunity.

Last year, I also spoke with Jeff Olson, VP Sales GreenButton at the HP Discover event in Las Vegas.

To try out this unique offering from GreenButton and HP Cloud, obtain a 90 day trial account from HP Cloud and contact Jeff Olsen from GreenButton to co-ordinate. If you have any additional HP Cloud questions, please contact our enterprise support team or myself via email or @hpcloud.

I also encourage you to check out this previous post on HP’s Cloud blog on how GreenButton simplifies Hadoop.

Posted in GreenButton, HP | Leave a comment

GreenButton Announces Partnership With NVIDIA To Support mental ray Renderer In The Cloud

GreenButton Works with NVIDIA to Provide mental ray Renderer in the Cloud

SAN JOSE, CALIF. – JANUARY 15, 2014 – GreenButton™, a leading provider of compute intensive and integrated on-demand cloud solutions, today announced the availability of NVIDIA’s mental ray renderer as an important addition to its existing offerings for the Digital Media and Entertainment industry.

The Academy Award® winning mental ray® is the most widely available production rendering solution in the world, and the native renderer within the Autodesk® Maya®, 3ds Max®, and Softimage® product suites. With mental ray in the cloud through GreenButton, users of Maya, 3ds Max and Softimage can now access massive computing resources on-demand at an extremely affordable cost.

“Working with NVIDIA allows us to address a bigger slice of the digital media and entertainment industry, a key market for us,” said Scott Houston, chief executive officer of GreenButton. “We are thrilled that together we can provide a more complete service for larger and smaller independent studios alike.”

The mental ray rendering service provides a multitude of benefits for customers, including the ability to access massive computing power on-demand with no upfront investment. In the world of VFX and computer graphics, this opens up the opportunity for users to compete in a market of creative resources and increasing business demands. Freelancers and small houses can now compete for the same jobs as larger studios and companies.

In addition, users have the ability to consume the very valuable mental ray licenses on a per-use basis. All computing time and license use is metered to the millisecond, providing an incredibly cost-effective solution. With the service, small to medium studios can gain access to tens of thousands of cores that would otherwise be unattainable. Larger studios can use the service to accommodate peak periods without over provisioning.

GreenButton’s Cloud Fabric platform provides the cloud access (via a plugin in Maya and 3dsMax), job distribution and execution, billing support and license management that the service depends upon. GreenButton Mission Control provides a management dashboard for the service as well as comprehensive reporting – such as usage, budgeting and job progress. In addition, the GreenButton CloudSync capability adds a critical component for those large and complex datasets that need to be synchronized with the cloud prior to rendering, with the added bonus of GridFTP support for accelerated transfers using the UDP protocol.

“As the rendering quality of productions continually rises, there’s an ever growing need for more processing power to satisfy it,” said Phillip Miller, director of Advanced Rendering Products at NVIDIA. “Our work to make mental ray available in the cloud with GreenButton allows artists to have the rendering power of a major studio, so their creative vision isn’t constrained by local resources.”

Visit the mental ray application page.

Posted in Cloud, GreenButton, Media | Leave a comment

Mass Compute On-Demand with V-Ray in the Cloud

Live! Wednesday, January 15, 2014 — 11:00am PDT

In today’s world of computer graphics and VFX, business demands are increasingly stringent to meet tight deadlines and deliver higher quality designs. This presents multiple challenges for artists since it is critical to access enough compute resources in order to address these demands or bid for new jobs. In this webinar, we will demonstrate how easy it can be for artists today to do this. We will demonstrate this using the V-Ray for 3ds Max plug-in from GreenButton.

The webinar is free and will last about about 1 hour, including the Q & A session.

Who is Presenting -
Sergio Gonzalez is a Solution Architect at GreenButton, Inc. He has worked in the visual effects industry for 7 years and as a Digital Media educator/freelancer for 12 years.

Where to Get Product Info –
For more info about GreenButton Voucher $100 visit the product page on the Novedge website.

Who Should Attend –
Users of V-Ray in a 3ds Max environment, users of other renderers interested in learning how they can easily access to mass compute

Registration –
Webinar is free but space is limited. Mark the date (Jan 15, 2014) and reserve your webinar seat now at:

gotomeeting.com/register/417919774

Posted in GreenButton | Leave a comment

Simplifying Hadoop

Check out this new blog post on our developer website written by GreenButton Solutions Architect, Christian Smith, on “Simplifying Hadoop”

“As the quantity and complexity of unstructured data increases, so does the need to process it. Businesses are finding clever and innovative ways of turning this data into a source of revenue, not directly, but through a better understanding of their business, their customers and the habits exhibited by them.

Because of this growth, we’re often finding the traditional data analysis techniques aren’t up to the task and as a result of this Hadoop has become a leader in the Big Data and Analytics spaces.

While the Hadoop ecosystem offers many valuable tools like MapReduce, HBase, Hive, PIG and Oozie, setting these up and running your jobs can be a daunting task. The Hadoop components require significant capital for the hardware and ops, and raise numerous questions like ‘how much hardware do I need?’ It would be nearly impossible to cater for all the potential workloads, and if you did you would probably end up with a lot of idle resources. If you aim too low, you end up with contention over the limited resources. What businesses really need is a managed Hadoop service that scales to their needs, enabling them to use the resources required for a particular job. This approach also mitigates the contention and fighting over resources. Need to run more than one job? No problem, just create a new cluster or scale up an existing one. When you’re done, you can just delete the cluster.”

This post also appears on HP’s Cloud Blog:
http://www.hpcloud.com/blog/greenbutton-simplifies-hadoop

Posted in GreenButton | Leave a comment

GreenButton Launches WarpDrive for Cloud Data Transfers

GreenButton Launches WarpDrive for Cloud Data Transfers
GreenButton™ on Thursday announced GreenButton WarpDrive, a solution providing accelerated and secure transfers of large datasets to the cloud. Previously this capability was only available to GreenButton customers for running Big Compute and Big Data workloads; however, demand from other use cases, such as enterprise backups, has led to a new standalone service.
“WarpDrive provides the best performing UDP data transfer technology for moving large datasets to the cloud.” said Dave Fellows, Chief Technology Officer at GreenButton. “UDP is far more efficient than TCP over high latency and congested networks. The ability to deliver this capability for a fraction of the cost of what’s currently on the market should open the door for many exciting cloud-based initiatives.”

Posted in GreenButton | Leave a comment

GreenButton inCus delivers cutting edge media solution

Audio & Visual Media is the centerpiece of almost all businesses and organizations today. In the olden days, multi-track masters were recorded on audio tapes and kept in vaults, then came the days of storing the same content on portable media digitally. These recordings were then typically stored in book cases, file cabinets, drawers and on media that can easily become out-of-date or lost. With exponential increase in generated media, keeping it effectively archived and ready for re-use can be an incremental problem. In addition, what if you wanted to search this stockpile? You would have to look through thousands of CDs, Hard drives or online storage and literally go through every single file to be able to find what you want. This renders the media you have practically unusable, worthless or at the least vastly time consuming.

Organizations today are having a hard time coming up with ideas to make archived media useful to their business once again, and actually generate more revenue or get considerable value out of the media they currently store. Even the simple task of searching through the archived content is a problem as they don’t have any way of say going back 10 years and looking at a specific piece of content based on a phrase or keyword actually spoken by someone. The typical approach for finding audio is to enter some textual information into a search field which then returns a list of media with a summarized textual descriptions. This only gets them half of the way, as they would still have to sit through the actual media content to find the specific point-in-time the user is after.

There are various solutions out there that index media files and have them available searchable using timed text files. This is all well and good, but most of these solutions have human interaction to actually transcribe the files which takes quite a while when we’re talking about hundreds of thousands of hours of content if not millions of hours. This time also increases and the quality of transcriptions decreases when the human interaction is outsourced to a non-English speaking country because it costs a fraction of the price it usually takes to transcribe using professional transcribers or stenographers.

So by now we know there are several problems plaguing the media industry today from a robust solution to archive the content all the way to searching, not just the meta-data of the content, but within the content itself. With the role of multimedia continuing to grow in the enterprise, Government and the internet, the need for technologies that better enable discovery and search of such content becomes all the more important.

This is where GreenButton inCus comes in. It’s a cutting edge audio indexing cloud-based service that provides optimized search and discovery experience for large stockpiles of media content. The service, at its core, is powered by MAVIS (Microsoft Research Audio Video Indexing Service) which has been in development for decades and is a patented piece of technology. Its secret sauce is state of the art Deep Neural Net based speech recognition technology which enables searching of audio or video files. What makes this cutting edge is that the service reduces errors in speech recognition by automatically expanding its vocabulary and storing word alternatives using a technique referred to as Probabilistic Word-Lattice Indexing. These techniques help provide highly accurate search results without any human interaction.

GreenButton inCus currently serves some high profile customers such as NASA with their NASA Tube, Department of Energy with their ScienceCinema & the British Library with Searching Speech (second largest library in the world). It’s also currently used in a vast array of verticals like Legal (JAVS & Verbatim for indexing court proceedings), Education, Archiving, Government Sectors, Social Media & Broadcasting. It’s a highly flexible and customizable service that can be accessed by UI tools or a flexible REST API developed by GreenButton. There are multiple moving parts to the solution that can be on premise or on the cloud. It’s as easy as signing up and using the service on demand at https://incus.greenbutton.com. You can read more about the service at http://www.greenbutton.com/incus or email us directly at sales@greenbutton.com.

Posted in Cloud, GreenButton, inCus, Media, Microsoft | Leave a comment

GreenButton Is Selected as Fast Growth Company and Vodafone “Ten Hot Emerging Company”

GreenButton is excited to announce that we have been selected as a fast growth company in the TIN100 Report issued by the TIN Network which monitors the performance of “New Zealand’s 200” (TIN100 and TIN100+). The TIN 100 represents the largest technology exporters in Information and Communications Technology (ICT), High Tech Manufacturing, and Biotechnology. This annual report is produced by Technology Investment Network (TIN) in association with Callaghan Innovation.

TIN Report Managing Director, Greg Shanahan said, “New Zealand’s technology export sector is buoyed by double digit (16.5%) growth from TIN100 ICT companies. From 2012-13, TIN100 companies increased their combined revenue by 3.7%, and the TIN100+ companies grew by 9%.” GreenButton listed in the TIN100+ report, is also a winner in the ‘Vodefone Ten Hot Emerging Company List.’

Scott Houston, Chief Executive Officer, GreenButton said, “GreenButton is excited to be selected as one of the TIN100+ and a Vodafone Ten Hot Emerging Company. Our experienced engineers create amazing cloud based solutions that enable our customers to affordably run HPC applications in the cloud,” Scott additionally stated, “our flagship solution, GreenButton Cloud Fabric, is adopted by companies around the world and supported on well-known platforms including Windows Azure, HP Cloud, OpenStack, VMware, AWS, and Profit Bricks.”

The report covers software sectors including healthcare, financial services and digital media. The report also states that cloud computing – GreenButton’s main market segment – is creating new growth and profit opportunities for their customers.

About the TIN100 Report and TIN100+ 10 Hot Emerging Companies

The purpose of the TIN100 Report is to quantify the economic significance of New Zealand’s globally-focused technology sector by capturing the key data on the country’s top 200 local high-tech companies. The Vodafone “Ten Hot Emerging Companies” list is made up of the TIN100+ companies with the highest dollar value growth. Only companies that have confirmed financial information and have achieved a growth rate of over 5% are considered for this list.

Posted in GreenButton | Leave a comment

Why Windows Azure?

A lot of our customers – including large Enterprises and Software Vendors – ask our opinion on the best cloud platform to run their applications and workloads. Two to three years ago there was only one decision to make; to Cloud or not to Cloud? Amazon Web Services was the defacto standard for those pioneers who chose Cloud. Fast forward to today, the decision to use the cloud is now the no-brainer for most businesses, however, with the cloud market becoming such a fiercely competitive space, ripe with innovation and differing strategies, the decision of which cloud becomes the more critical and daunting one.

When I read analyst opinion comparing the major cloud offerings, I’m surprised to see Windows Azure is often omitted. This leads me to believe these analysts are clearly not doing their homework or, they don’t have a deep understanding of the cloud and how this relates to the challenges businesses are faced with today.

Amazon Web Services are numero uno right now – that’s not up for debate. However, I see Windows Azure as the clear runner up and hot on Amazon’s heels. My prediction? Hold on to your seats for this one. I predict that Windows Azure will take the lead in 2-3 years’ time.

The momentum of Windows Azure is truly impressive. Microsoft, who have been criticized in the past for moving too slowly to capitalize on industry trends, have really proven the naysayers wrong with the speed and agility in which they have innovated and improved their cloud offering. If you look at Scott Guthrie’s blog, which is a good yardstick for the pace of Azure development, there have been 18 major announcements on new Azure features in the last 7 months! The rate of progress with the IaaS offering has been particularly notable. Scott Guthrie, affectionately known as “The Gu”, is the VP of Windows Azure and is essentially a god to Microsoft developers. While Scott has clearly made a monumental impact to Windows Azure and demonstrated impressive leadership since being promoted to run Azure in 2011, there is also a team of incredibly smart and visionary engineers that have designed and built a platform that is second to none. Yes you heard me right, putting market share aside, I believe Windows Azure is the most complete and capable offering in the market today. Big call huh?

Aside from the many commercial and enterprise-focused features (Backup services, Active Directory, Managed SQL Server, Private-Public cloud integration, Hadoop as a Service etc), that make Azure a natural fit for most organizations, we at GreenButton are excited by some of the lesser-known capabilities. The GreenButton Cloud Fabric platform for cloud-enabling and managing Big Compute and Big Data apps can place heavy demands on infrastructure, particularly when you look at the more traditional High Performance Computing (HPC) workloads such as Seismic Processing, Computational Fluid Dynamics and Genomic Sequencing algorithms. Many of these applications require extremely low latency networks to join many computers together in a cluster. The expensive network configurations are typically built around Infiniband and use Remote Direct Memory Access (RDMA) to provide around 2 micro seconds latency. None of the major cloud providers support Infiniband today. Microsoft however announced their new HPC hardware complete with Infiniband a little while ago and we expect to see this coming online soon. The cloud has long been perceived by many of the HPC traditionalists as an inappropriate or ill-equipped alternative for High Performance Technical Computing – particularly for those workloads that require very low latency networks. This perception is rapidly changing as both Commercial and Research & Science facilities are finding the massive economic benefits the cloud offers for these spikey-by-nature workloads hard to resist. With this latest offering, Microsoft claim a real leadership position in this space. Of course those with HPC applications will want to talk to us to get their workloads ported to the cloud and manage data and governance without any fuss…

We have other strategic partners who certainly have their strengths. HP are a late starter but making notable progress on articulating a thought out strategy. IBM shouldn’t be underestimated with their acquisition of SoftLayer and their High Performance Computing and Analytics pedigree. VMware still dominate the Private Cloud and have a seamless Hybrid Cloud offering now. But I see Microsoft best positioned to make the most impact in the near term.

Microsoft is in the process of shifting to a more functional organization across Devices & Consumer and Commercial. Within those groups we should expect to see more collaboration and more tightly integrated products and services. This is an exciting transition and one which should deliver a more connected and seamless experience to customers.

Microsoft have demonstrated a strategy towards offering vertically integrated services. Windows Azure Media Services is a good example. I think this is bang on from a strategy point of view; Customers want higher-value services that are consumed on-demand. They also want these services to be connected with their existing ecosystems. This is surely the end goal of all this cloud malarkey.

It’s an incredibly exciting industry to be a part of right now. Our vision has always been to make the power of on-demand computing available to anyone at the push of a button – whether you’re a “one-man-band” or a Fortune 500. To consumerize High Performance Computing. To allow the little guy to compete with the largest in their industry without constraints. Microsoft and Windows Azure are playing a key role in the realization of this disruptive vision.

Posted in Cloud, HPC, Microsoft | Tagged , , , , , | 6 Comments