Now you should be able to browse to Jupyter using your workstation’s browser, nice and straightforward. Ssh -L 8888:$UK8S_IP:$J_PORT ssh Landing on Jupyter: testing our Spark cluster from a Jupyter notebook Yep, it’s time to open another terminal and run the following commands so we can set up a tunnel to help us to do that: GCE_IIP=$(gcloud compute instances list | grep ubuntu-core-20 | awk '') We will use an SSH tunnel to push the image to our remote private registry on MicroK8s.
#INSTALL APACHE SPARK ON EC2 UBUNTU INSTALL#
Let’s build that container image so that we can launch it on our Ubuntu Core hosted MicroK8s in the sky: sudo apt install docker.io So Apache Spark runs in OCI containers on Kubernetes. # Specify the User that the actual main process will run asĮOF Containers: the hottest thing to hit 2009 Call to hacktion: adapting the Spark Dockerfileįirst, let’s download some Apache Spark release binaries and adapt the dockerfile so that it plays nicely with pyspark: wget Ĭhgrp root /etc/passwd & chmod ug+rw /etc/passwd & \ĬOPY kubernetes/dockerfiles/spark/entrypoint.sh /opt/ĬOPY kubernetes/dockerfiles/spark/decom.sh /opt/ Let’s take a look at getting Apache Spark on this thing so we can do all the data scientist stuff. If so, you can exit any SSH session to your Ubuntu Core in the sky and return to your local system. If you’ve followed the steps in Part 1 and Part 2 of this series, you’ll have a working MicroK8s on the next-gen Ubuntu Core OS deployed, up, and running on the cloud with nested virtualisation using LXD.
0 kommentar(er)
