Apache Zeppelin (with Apache Spark on YARN over Pseudo Distributed Hadoop)
2.1K
Dockerfile linksApache Zeppelin 0.10.1 | Apache Spark 3.1.3
Apache Zeppelin 0.10.0 | Apache Spark 3.1.2
docker run --rm -d\
--name zeppelin-spark-pseudo\
--hostname zeppelin-spark-pseudo\
--env ZEPPELIN_ADDR=0.0.0.0\
--env ZEPPELIN_PORT=18888\
--env ZEPPELIN_INTERPRETER_DEP_MVNREPO=https://repo1.maven.org/maven2/\
--env SPARK_HOME=/opt/spark\
--env ZEPPELIN_INTERPRETER_CONNECT_TIMEOUT=120000\
--env YARN_SITE__YARN_NODEMANAGER_RESOURCE_DETECT_HARDWARE_CAPABILITIES=true\
--publish 7077:7077\
--publish 8032:8032\
--publish 8088:8088\
--publish 8042:8042\
--publish 18080:18080\
--publish 18888:18888\
loum/zeppelin-spark-pseudo:latest
Zeppelin web UI is available at http://localhost:18888
NOTE: there's quite a bit happening here so depending on your local setup, you will have to give the services some time to stabilise. A simple check is to run the following:
docker logs --follow zeppelin-spark-pseudo... and look for the following output (previous logging omitted for brevity):
... ### Starting the Spark cluster ... starting org.apache.spark.deploy.master.Master, logging to /tmp/spark/spark--org.apache.spark.deploy.master.Master-1-zeppelin-spark-pseudo.out localhost: starting org.apache.spark.deploy.worker.Worker, logging to /tmp/spark/spark-hdfs-org.apache.spark.deploy.worker.Worker-1-zeppelin-spark-pseudo.outRunning the Zeppelin Spark interpreter before the underlying Spark service is available will crash the interpreter. If that happens, restart the Zeppelin Spark interpreter and wait until Spark service is available.
NOTE: many of the default Zeppelin tutes are super-dodge. I have tried to add my own but all the keeping all versions free of conflict around the dated Zeppelin wrappers is tedious. I'm looking for alternatives ...
docker stop zeppelin-spark-pseudo
Content type
Image
Digest
sha256:89c5ae91c…
Size
3.2 GB
Last updated
almost 4 years ago
docker pull loum/zeppelin-spark-pseudo