The Performance Beacon

The web performance, analytics, and optimization blog

SOASTA brings Julia to Big Data, with a real user twist

This week at SOASTA: We headed east to Big Data TechCon in Boston, stopped by MIT (the home of Julia and the Julia Project), and launched a contest where you can earn a cool data science tshirt.

First stop: Big Data TechCon

I emceed a great session led by SOASTA chair Ken Gardner (@kennethcgardner) and one of our chief product design architects, Philip Tellis (@bluesmoon), founder of Boomerang and LogNormal.

The SOASTA session delved deep into the technical trade-offs and selection process involved in choosing the underlying architecture for SOASTA’s Data Science WorkBench (DSWB). DSWB is an extension of our real user measurement (RUM) solution, mPulse. mPulse presents real user beacon data that is collected from web and mobile users by the billions every second of every day. This beacon data can then be used for marketing campaigns, performance enhancements and any other types of data analysis required by a business to use the information to increase bottom line revenue — the ultimate Big Data application. (Check out Tammy Everts’s recent blog post to get a deeper dive into mPulse and see the visualizations created with Julia in the DSWB using mPulse data.  You’ll be hooked.)

Ken discussed how some of today’s advanced technologies that are available, particularly in the open source area, have opened the doors for building a data science platform including the required infrastructure, the handling of the data pipeline and the analysis and workflow. Oh, and the cost is extremely affordable, especially when compared to the other options offered in the marketplace.

Ken Gardner at Big Data TechCon

Ken’s talk specifically delved into the Julia language, and the alternatives considered, including R and Python.  In addition, Ken discussed the choice of Amazon Redshift as the data warehouse platform for the DSWB, and why this architecture was chosen over the other alternatives considered, such as Hadoop and Big Query.

At this point, our session then gave the attendees a hands-on workshop to “play along” with the DSWB as Philip Tellis led the attendees in a Julia tutorial where they created and ran function calls created in Julia against “scrubbed” real user data from mPulse via the data warehouse in Amazon’s Redshift.

Next stop: MIT!

Once the conference came to a close, our focus shifted over to Julia’s home, the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL). Ken, Philip and I paid a visit to Jiahao Chen (@acidflask), research scientist for Julia, and his team at the CSAIL.

In keeping with SOASTA’s leadership in technology and in the marketplace since its inception in 2006, SOASTA is leaps and bounds ahead in this marketplace with the selection of Julia, as evidenced while visiting the Julia team.

In an e-mail to us after our visit, Jiahao Chen said, “Thanks again for dropping by MIT. It was really great to hear about how Julia is being used to do real business, not just pretend work in an academic research group or university homework.”

SOASTA certainly looks forward to continuing engagement with Julia, and we will see the Julia team at JuliaCon 2015 in Boston, MA from June 24th to the 28th.

How to win your own data science tshirt

The SOASTA Data Science team also unveiled some cool swag. Since data science is red hot these days, we chose red as the color for our “SOASTA Data Scientist, Performance is Everything” t-shirt.  Nice, huh?

Win a SOASTA data science t-shirt

I asked our chair, Ken Gardner, what people have to do to get one.  The response was, “Write two Julia functions.” So as an ex-developer (okay, it’s been awhile, but it should be just like riding a bike, right?), I gave this challenge a shot.

I went over to the Julia site, downloaded and installed Julia, checked out GitHub, the online tutorials, documentation, sample functions, I/O, packages, etc.

What I came up with was to utilize the Julia package and I/O functions using the calendar.  Simple, right?

Here it is:


I then gave some function creation a spin. And it wouldn’t be a complete exercise if I didn’t come up with some creative functions, eh?


No errors!  GoGators!  (You didn’t think there’d be any errors with THAT function, did you?)

So, now that I’ve checked off the package, I/O and function creation features of Julia, I have one comment for Ken:  “I’ll take my shirt in a large, please!”

…after all, you didn’t say anything about having to clean compile it, check it in/out using Github, or any of the other features and steps.

You’re probably thinking, “Hey, how do I get one?”  Simple. Hop on board with mPulse and the Data Science Workbench (for free!), take our class and get certified, and tell us your size.

What are you waiting for?

Sign up for mPulse

Dan Boutin

About the Author

Dan Boutin

Dan is the Vice President of Digital Strategy for SOASTA. In this role, Dan is responsible taking the world's first Digital Performance Management (DPM) solution to market as a trusted advisor for SOASTA's strategic customers, and changing the way ecommerce organizations approach the marketplace.

Follow @DanBoutinGNV