Apache Spark, Spark + Cassandra, Cassandra 2.2/3.0

Speakers Bio:

Brendon Smith, Mayur Ladwa

Brendon and Mayur are software developers in BlackRock.

Brendon manages the investment risk team and is also responsible for delivering various fixed-income tools in BlackRock.

Mayur works in the core client processing team dealing with tools around the client investment lifecycle. Both of them are passionate about big data technologies and spend a lot of their own time learning new ways of solving data-related challenges.

Christopher Batey (@chbatey)

Christopher Batey is a Software Engineer by trade and is currently employed by DataStax as a Technical Evangelist for Apache Cassandra. Chris has also worked for Sky, where he helped build their online television platform, and IBM, where he helped develop a variety of messaging products. He spends a lot of his own time contributing back to the software community. He’s founder of an open source test double for Apache Cassandra: Stubbed Cassandra, helps with the running of the London Java Community and blogs regularly at: http://christopher-batey.blogspot.co.uk/

Meetup Summary:

Historically, a majority of companies have been implementing various data warehouse solutions, possibly with structured data in a star schema environment. Given the increasing amount of data and performance benefits of newer technologies, in this talk Brendon and Mayur walked through how we use technologies such as Spark and a HDFS compliant file system with columnar storage for a multi-data center architecture that allows employees firm wide to run analytical queries at BlackRock.

Cassandra 2.2-rc2 has just been released which means that 2.2 is just around the corner. This talk also highlighted the new features of this version and how they affect the way you model data in C*. The presenters discussed various aspects of the following topics:

  • JSON support
  • User defined functions
  • User defined aggregates
  • Role based authentication

They also stepped into the future and took a look at 3.0 with Materialized Views being the major highlight.

For more photos of the session, please check out this event’s meetup page here!

BlackRock Engineering