About This Book

Develop a machine learning system with Spark's MLlib and scalable algorithmsDeploy Spark jobs to various clusters such as Mesos, EC2, Chef, YARN, EMR, and so onThis is a step-by-step tutorial that unleashes the power of Spark and its latest features

Who This Book Is For

Fast Data Processing with Spark - Second Edition is for software developers who want to learn how to write distributed programs with Spark. It will help developers who have had problems that were too big to be dealt with on a single computer. No previous experience with distributed programming is necessary. This book assumes knowledge of either Java, Scala, or Python.

What You Will Learn

Install and set up Spark on your cluster Prototype distributed applications with Spark's interactive shell Learn different ways to interact with Spark's distributed representation of data (RDDs) Query Spark with a SQL-like query syntax Effectively test your distributed software Recognize how Spark works with big data Implement machine learning systems with highly scalable algorithms

In Detail

Spark is a framework used for writing fast, distributed programs. Spark solves similar problems as Hadoop MapReduce does, but with a fast in-memory approach and a clean functional style API. With its ability to integrate with Hadoop and built-in tools for interactive query analysis (Spark SQL), large-scale graph processing and analysis (GraphX), and real-time analysis (Spark Streaming), it can be interactively used to quickly process and query big datasets.

Fast Data Processing with Spark - Second Edition covers how to write distributed programs with Spark. The book will guide you through every step required to write effective distributed programs from setting up your cluster and interactively exploring the API to developing analytics applications and tuning them for your purposes.

About the Author

Krishna Sankar

Krishna Sankar is a chief data scientist at http://www.blackarrow.tv/, where he focuses on optimizing user experiences via inference, intelligence, and interfaces. His earlier roles include principal architect, data scientist at Tata America Intl, director of a data science and bioinformatics start-up, and a distinguished engineer at Cisco. He has spoken at various conferences, such as Strata-Sparkcamp, OSCON, Pycon, and Pydata about predicting NFL (http://goo.gl/movfds), Spark (http://goo.gl/E4kqMD), data science (http://goo.gl/9pyJMH), machine learning (http://goo.gl/SXF53n), and social media analysis (http://goo.gl/D9YpVQ). He was a guest lecturer at Naval Postgraduate School, Monterey. His blogs can be found at https://doubleclix.wordpress.com/. His other passion is Lego Robotics. You can find him at the St. Louis FLL World Competition as the robots design judge.

Holden Karau

Holden Karau is a software development engineer and is active in the open source sphere. She has worked on a variety of search, classification, and distributed systems problems at Databricks, Google, Foursquare, and Amazon. She graduated from the University of Waterloo with a bachelor's of mathematics degree in computer science. Other than software, she enjoys playing with fire and hula hoops, and welding.