Databricks-Connect in a Container

Today I have published a set of containers to Docker Hub to enable developers to create Databricks-Connect Dev environments really quickly and consistently. Docker Hub: https://hub.docker.com/r/datathirstltd/dbconnect Source Code: https://github.com/DataThirstLtd/databricksConnectDocker These are targeted at dev using PySpark in VSCode. Though I suspect it will work for Scala and Java development as well. Why? Because setting up Databricks-Connect (particularly on Windows is a PIA). This allow: A common setup between team members Multiple side by side versions Ability to reset your environment Even run the whole thing from a browser! [Read More]

PySpark in Visual Studio Online (Container)

A new product was recently released called Visual Studio Online (VSOnline). Not to be confused with the old Visual Studio Online (VSO) which was renamed VSTS and then renamed to the now Azure DevOps (that’s a rant for another day). Update May 2020: Microsoft have renamed it already - its now CodeSpaces! I quite like that name. VSOnline allows you to create a development environment in a container hosted in Azure for almost pennies. [Read More]