Scala Project setup using 'sbt' & Eclipse

This post provides instruction to set up a new Scala Project using sbt build tool. We set up a new Project, Create a simple Application & also submit the Application to Spark

We would need to use both Command Line & Eclipse(Hope there is better way to do everything from Eclipse..but I could not find one)

Create Build Definition

Create a directory where you want to have your Scala Project. In my case, I have created a directory called sbook. In sbt terms this directory is called Base Directory

Create the build definition file build.sbt in Base Directory

lazy val root = (project in file(".")). 
  settings(
    name := "sparkexperiments",
    version := "1.0",  
    // Scala Version
    scalaVersion := "2.10.4",
    // Download source code(will come in handy to refer the code in eclipse)
    EclipseKeys.withSource := true,
    libraryDependencies ++= Seq(
     //'provided' - We do not want to ship these JARs to
     //             worker nodes, as they will be already available
     //             in the worked nodes
     "org.apache.spark" % "spark-core_2.10" % "1.5.2" % "provided"
    )
 )

Create build.properties that contains the sbt version(sbt --version) that is used to create this build
mountain@mountain:~/sbook$ cat build.properties 
sbt.version=0.13.8

Create Eclipse Configuration

Let us create Eclipse configuration , so that we can import the Project into Eclipse
mountain@mountain:~/sbook$ sbt eclipse

Running sbt eclipse also creates all the sbt recommended directory structure
mountain@mountain:~/sbook$ ls
build.properties  build.sbt  project  src  target

Now we can import the Project in Eclipse by using the option import -> Existing Projects into Workspace

Our First Scala Application

Let us create a Scala Application to print a String
sbook.helloworld.ScalaTest

package sbook.helloworld
object ScalaTest {
     def main(args: Array[String]) {
       println("My first Scala Application");
     }
}

Packaging the Application

We would need to package our application into a single Jar
mountain@mountain:~/sbook$ sbt package

This creates the Jar file in 'target' directory,
mountain@mountain:~/sbook$ find . -name *.jar
./target/scala-2.11/sparkbookapp_2.11-1.0.jar

Running the Application

Let us submit our first Application to Spark  using spark-submit script
mountain@mountain:~/sbook$ spark-submit --class sbook.helloworld.ScalaTest target/scala-2.11/sparkbookapp_2.11-1.0.jar 
My first Scala Application

That is the end of our First Hello World Application

Reference

http://www.scala-sbt.org/0.13/tutorial/Hello.html