Official Docker image: Dockerfile and Github Actions workflow (#1945)

* Add Dockerfile and Github Actions workflow for Official Docker image

* Update Dockerfile

Co-authored-by: Barry Hart <barrywhart@yahoo.com>

* Update docker build comments

* Fix spelling mistake

* Official docker test (#2)

* Refactor Dockerfile for better caching performance; add .dockerignore.

* Implement layer caching in GHA.

Co-authored-by: Chris Curro <git@curro.cc>

* revert requirements.txt to be handled by #1956

* make docker image test consistent with suggested usage

* Add README reference to Official SQLFluff Docker Image

* Set separate end-user working directory for convenience

* remove -t option in integration test

* Add workflow_dispatch option incase we manually want to trigger docker build and push

* Extract dependencies from setup.cfg

Co-authored-by: Barry Hart <barrywhart@yahoo.com>
Co-authored-by: Chris Curro <git@curro.cc>

.dockerignore

@@ -0,0 +1,54 @@
+# Ignore IDE files
+.vscode
+.idea
+/.sqlfluff
+**/.DS_Store
+
+# Ignore Python cache and prebuilt things
+.cache
+__pycache__
+*.egg-info
+*.pyc
+build
+_build
+dist
+.pytest_cache
+
+# Ignore the Environment
+env
+.tox
+venv
+.venv
+.python-version
+
+# Ignore coverage reports
+.coverage
+.coverage.*
+coverage.xml
+htmlcov
+*.cover
+
+# Ignore test reports
+.test-reports
+test-reports
+
+# Ignore root testing sql & python files
+/test*.sql
+/test*.py
+/.hypothesis/
+
+# Ignore dbt outputs from testing
+/target
+
+# Ignore conda environment.yml contributors might be using and direnv config
+environment.yml
+.envrc
+**/*FIXED.sql
+
+# Others
+pip-log.txt
+pip-delete-this-directory.txt
+*.log
+.git
+.mypy_cache
+.pytest_cache

.github/workflows/publish-sqlfluff-docker-image-to-dockerhub.yaml

@@ -0,0 +1,70 @@
+# Create and push Docker image of latest release to DockerHub.
+name: Publish SQLFluff DockerHub Version
+on:
+  release:
+    types:
+      - published
+  workflow_dispatch:
+
+# Create tag for integration test.
+env:
+  TEST_TAG: ${{ secrets.DOCKERHUB_USERNAME }}/sqlfluff:test
+
+jobs:
+  docker:
+    runs-on: ubuntu-latest
+    steps:
+      # Get the version of latest release in
+      # order to tag published Docker image.
+      - name: Get latest release name
+        id: latest_release
+        uses: pozetroninc/github-action-get-latest-release@master
+        with:
+          repository: ${{ github.repository }}
+
+      # Setup QEMU and Buildx to allow for multi-platform builds.
+      - name: Set up QEMU
+        id: docker_qemu
+        uses: docker/setup-qemu-action@v1
+      - name: Set up Docker Buildx
+        id: docker_buildx
+        uses: docker/setup-buildx-action@v1
+
+      # Authenticate with DockerHub.
+      - name: Login to DockerHub
+        id: docker_login
+        uses: docker/login-action@v1 
+        with:
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+      
+      # Build amd64 image to use in the integration test.
+      - name: Build and export to Docker
+        id: docker_build
+        uses: docker/build-push-action@v2
+        with:
+          load: true
+          tags: ${{ env.TEST_TAG }}
+          cache-from: type=registry,ref=${{ secrets.DOCKERHUB_USERNAME }}/sqlfluff:latest
+          cache-to: type=inline
+
+      # Integration test to validate newly created image is working.
+      - name: Test Docker image
+        id: docker_test
+        run: |
+          echo "SELECT 1" > test.sql
+          docker run --rm -i -v $PWD:/sql ${{ env.TEST_TAG }} lint /sql/test.sql
+
+      # Build arm64 image (amd64 is cached from docker_build step) and export to DockerHub.
+      # N.B. We tag this image as both latest and with its version number.
+      - name: Build and push
+        id: docker_build_push
+        uses: docker/build-push-action@v2
+        with:
+          push: true
+          platforms: linux/amd64,linux/arm64
+          tags: |
+            ${{ secrets.DOCKERHUB_USERNAME }}/sqlfluff:latest
+            ${{ secrets.DOCKERHUB_USERNAME }}/sqlfluff:${{ steps.latest_release.outputs.release }}
+          cache-from: type=registry,ref=${{ secrets.DOCKERHUB_USERNAME }}/sqlfluff:latest
+          cache-to: type=inline

Dockerfile

@@ -0,0 +1,38 @@
+FROM python:3.9-slim-bullseye
+
+# Set separate working directory for easier debugging.
+WORKDIR /app
+
+# Create virtual environment.
+ENV VIRTUAL_ENV /app/.venv
+RUN python -m venv $VIRTUAL_ENV
+ENV PATH $VIRTUAL_ENV/bin:$PATH
+RUN pip install --upgrade pip setuptools wheel
+
+# Install requirements seperately
+# to take advantage of layer caching.
+# N.B. we extract the requirements from setup.cfg
+COPY setup.cfg .
+RUN python -c "import configparser; c = configparser.ConfigParser(); c.read('setup.cfg'); print(c['options']['install_requires'])" > requirements.txt
+RUN pip install --upgrade -r requirements.txt
+
+# Copy minimal set of SQLFluff package files.
+COPY MANIFEST.in .
+COPY README.md .
+COPY setup.py .
+COPY src ./src
+
+# Install sqlfluff package.
+RUN pip install --no-dependencies .
+
+# Switch to non-root user.
+USER 5000
+
+# Switch to new working directory as default bind mount location.
+# User can bind mount to /sql and not have to specify the full file path in the command:
+# i.e. docker run --rm -it -v $PWD:/sql sqlfluff/sqlfluff:latest lint test.sql
+WORKDIR /sql
+
+# Set SQLFluff command as entry point for image.
+ENTRYPOINT ["sqlfluff"]
+CMD ["--help"]

README.md

@@ -60,7 +60,7 @@ L:   1 | P:  14 | L006 | Operators should be surrounded by a single space unless
 L:   1 | P:  27 | L001 | Unnecessary trailing whitespace
 ```
 
-You can also have a play using [**SQLFluff online**](https://online.sqlfluff.com/).
+Alternatively, you can use the [**Official SQLFluff Docker Image**](https://hub.docker.com/repository/docker/sqlfluff/sqlfluff/general) or have a play using [**SQLFluff online**](https://online.sqlfluff.com/).
 
 For full [CLI usage](https://docs.sqlfluff.com/en/stable/cli.html) and [rules reference](https://docs.sqlfluff.com/en/stable/rules.html), see [the SQLFluff docs](https://docs.sqlfluff.com/en/stable/).