At Data Thirst we are committed to building tools to assist everyone in implementing technology. We have seen similar challenges on many different projects, and therefore decided to produce tools to help solve these problems. We open them for everyone to use and to help build them out further.

CI/CD PowerShell Module for Databricks

This PowerShell module has been created a wrapper to the REST API offered by Databricks. The REST API is actually quite complex to use, so this wrapper is to make common tasks simple. You can upload files to DBFS, deploy (import and export) notebooks, manage clusters, job & libraries. The tools are the basis to building your automated deployment pipelines.

They are Open Source on GitHub here and published to the PowerShell Gallery here.

Documentation can be found on the GitHub Wiki.

Azure DevOps Extension for Databricks

Based upon the PowerShell module above this extension to VSTS/Azure DevOps allows you to deploy notebooks from source control, deploy secrets, and deploy files to DBFS without having to write any code.

The extension is available in the Azure DevOps MarketPlace for FREE here.

DBFS File Explorer for Databricks

DBFS Explorer is a tool for uploading and downloading files to the Databricks filesystem from your local desktop. More information and downloads are here.