We can monitor our Spark Application using the Web App provided by Spark. The App is accessible at port 4040. If we are running multiple Application, then successive ports(4041, ...) will be assigned to each Application. We can only access the Web App during its execution. Once the Application is complete, the Web App will no longer be accessible
Let us go for a demo,
Start an Application. For this demo, Scala shell acts as a Driver(Application)
mountain@mountain:~/sbook$ spark-shell
Connect to web app(localhost:4040) and here we have the Web App to monitor our Application
Let us go for a demo,
Start an Application. For this demo, Scala shell acts as a Driver(Application)
mountain@mountain:~/sbook$ spark-shell
Connect to web app(localhost:4040) and here we have the Web App to monitor our Application