06 December 2015

parallelize() Examples

scala> //for loop Example
scala> val rdd = sc.parallelize(for {
     |     x &lt- 1 to 3
     |     y &lt- 1 to 2
     | } yield (x, None))
rdd: org.apache.spark.rdd.RDD[(Int, None.type)] = ParallelCollectionRDD[38] at parallelize at :21

scala>

scala> rdd.collect()
res15: Array[(Int, None.type)] = Array((1,None), (1,None), (2,None), (2,None), (3,None), (3,None))

Reference

for loop with yieldhttp://alvinalexander.com/scala/scala-for-loop-yield-examples-yield-tutorial