Databricks-Connect in a Container
Today I have published a set of containers to Docker Hub to enable developers to create Databricks-Connect Dev environments really quickly and consistently.
Docker Hub: https://hub.docker.com/r/datathirstltd/dbconnect Source Code: https://github.com/DataThirstLtd/databricksConnectDocker These are targeted at dev using PySpark in VSCode. Though I suspect it will work for Scala and Java development as well.
Why? Because setting up Databricks-Connect (particularly on Windows is a PIA). This allow:
A common setup between team members Multiple side by side versions Ability to reset your environment Even run the whole thing from a browser!
[Read More]