Last week I had the great pleasure of attending #MesosCon in Seattle. Mesos is a technology that I had heard some things about, and was interested in seeing how we could apply it to our systems here at People Pattern. I went knowing that, in a nutshell, Mesos allows you to take a pool of resources and aggregate them together, then allocate portions of the combined cpu, memory, and storage to jobs or processes. What I experienced at #MesosCon has me really excited about the things to come in the world of big data infrastructure and DevOps.
Before I get into what Mesos is and some of the things you can do with it, I want to point out: There is some serious backing behind this platform. Some of the presenters at the conference who showcased how they use Mesos include Twitter, eBay, Netflix, Hubspot, and Apple. That’s right. Apple’s Siri runs on Mesos.
So, what is Mesos?
Check out this diagram of a simple Mesos cluster that I’ve shamelessly stolen from here:
I’m a little squeamish about the “master” and “slave” terminology too. There’s good news about this. At the conference, the core team made a big issue of emphasizing that going forward in the 1.0 release the terminology is changing from “slave” to “agent”. After 1.0, protocols will support both terms until the planned deprecation of the old terminology with the 2.0 release. Moving on.
It’s a beautifully simple system. A cluster of zookeeper instances elects a Mesos master via a quorum. That master offers up the combined resources of all associated agents to various schedulers that can run on the Mesos cluster itself. Schedulers run batch jobs or maintain long running processes, providing a web and/or API interface to resources and running jobs. A few of the schedulers that were popular among users at the conference include marathon, chronos, and aurora. Smartly, there’s a relatively new service provided by Mesosphere’s DCOS that rolls all of this up and makes it super easy to deploy to the cloud or on your own infrastructure. I know, right? Awesome!
Once you have your Mesos cluster, what can you do with it?
Out of the box, Mesos comes with its own native Mesos Containerizer, added in 0.20.0 a Docker Containerizer (that’s right!), and a framework for adding your own External Containerizers. Spelling it out, this means anything you can build into a docker container, you can run on Mesos. For service discovery, Mesos-DNS seems to be a popular option.
In addition to what you can deploy of your own code via containers, the Mesos community has been busy at work developing support for the most in-demand applications. Elasticsearch, Kibana, Logstash, Kafka, the Spark Stack, Hadoop, Kubernetes, YARN, all run on Mesos and have the backing of the community. This is a very big deal.
Is it really that simple?
Yes, absolutely. Or at least I think so. I’ll do another blog post once I’ve implemented some of this. Let’s assume it is, okay? So, Mesos is awesome. Now that the Kool-Aid has been offered and received with wild enthusiasm, what were some of the more cautionary or prescriptive take aways from the conference?
Check out my next post to find out! In the meantime, feel free to check out a demo of the platform.
Recent Comments