Compare commits

...

4 Commits

Author SHA1 Message Date
Github Build Bot
ee6a69a7c3 Bumping version to 1.11.0rc1 and generate changelog 2025-11-18 18:49:22 +00:00
github-actions[bot]
0d59a3d5a4 Fix 10157: macro composition in unit tests (#12168) (#12182)
(cherry picked from commit f6e0793d00)

Co-authored-by: Michelle Ark <MichelleArk@users.noreply.github.com>
2025-11-18 13:45:13 -05:00
github-actions[bot]
b52eb6f8e7 Support default arguments for udfs (#12175) (#12181)
* Add tests to check parsing of function argument default values

* Begin allowing the specification of `default_value` on function arguments

* Validate that non-default function arguments don't come _after_ default function arguments

* Add changie doc

(cherry picked from commit 811e4ee955)

Co-authored-by: Quigley Malcolm <QMalcolm@users.noreply.github.com>
2025-11-18 12:33:53 -06:00
Emily Rockman
8666c83f26 move full workflow into core since no one else uses it 2025-11-18 09:55:58 -05:00
22 changed files with 707 additions and 30 deletions

View File

@@ -1,5 +1,5 @@
[bumpversion]
current_version = 1.11.0b4
current_version = 1.11.0rc1
parse = (?P<major>[\d]+) # major version number
\.(?P<minor>[\d]+) # minor version number
\.(?P<patch>[\d]+) # patch version number

28
.changes/1.11.0-rc1.md Normal file
View File

@@ -0,0 +1,28 @@
## dbt-core 1.11.0-rc1 - November 18, 2025
### Features
- Allow for defining funciton arguments with default values ([#12044](https://github.com/dbt-labs/dbt-core/issues/12044))
### Fixes
- Fix parse error when build_after.count set to 0 ([#12136](https://github.com/dbt-labs/dbt-core/issues/12136))
- Stop compiling python udfs like python models ([#12153](https://github.com/dbt-labs/dbt-core/issues/12153))
- For metric names, fix bug allowing hyphens (not allowed in metricflow already), make validation throw ValidationErrors (not ParsingErrors), and add tests. ([#n/a](https://github.com/dbt-labs/dbt-core/issues/n/a))
- Include macros in unit test parsing ([#10157](https://github.com/dbt-labs/dbt-core/issues/10157))
### Under the Hood
- add dbt/jsonschemas to manifest.in ([#12126](https://github.com/dbt-labs/dbt-core/issues/12126))
- Move from setup.py to pyproject.toml ([#5696](https://github.com/dbt-labs/dbt-core/issues/5696))
- Fixes issue where config isn't propagated to metric from measure when set as create_metric:True ([#None](https://github.com/dbt-labs/dbt-core/issues/None))
- Support DBT_ENGINE prefix for record-mode env vars ([#12149](https://github.com/dbt-labs/dbt-core/issues/12149))
### Dependencies
- Drop support for python 3.9 ([#12118](https://github.com/dbt-labs/dbt-core/issues/12118))
### Contributors
- [@WilliamDee](https://github.com/WilliamDee) ([#None](https://github.com/dbt-labs/dbt-core/issues/None))
- [@nathanskone](https://github.com/nathanskone) ([#10157](https://github.com/dbt-labs/dbt-core/issues/10157))
- [@theyostalservice](https://github.com/theyostalservice) ([#n/a](https://github.com/dbt-labs/dbt-core/issues/n/a))

View File

@@ -0,0 +1,6 @@
kind: Features
body: Allow for defining funciton arguments with default values
time: 2025-11-17T14:10:53.860178-06:00
custom:
Author: QMalcolm
Issue: "12044"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Include macros in unit test parsing
time: 2025-11-17T14:06:49.518566-05:00
custom:
Author: michelleark nathanskone
Issue: "10157"

View File

@@ -1,25 +1,44 @@
# **what?**
# Cuts a new `*.latest` branch
# Also cleans up all files in `.changes/unreleased` and `.changes/previous verion on
# `main` and bumps `main` to the input version.
# Cuts the `*.latest` branch, bumps dependencies on it, cleans up all files in `.changes/unreleased`
# and `.changes/previous verion on main and bumps main to the input version.
# **why?**
# Generally reduces the workload of engineers and reduces error. Allow automation.
# Clean up the main branch after a release branch is cut and automate cutting the release branch.
# Generally reduces the workload of engineers and reducing error.
# **when?**
# This will run when called manually.
# This will run when called manually or when triggered in another workflow.
# Example Usage including required permissions: TODO: update once finalized
# permissions:
# contents: read
# pull-requests: write
#
# name: Cut Release Branch
# jobs:
# changelog:
# uses: dbt-labs/actions/.github/workflows/cut-release-branch.yml@main
# with:
# new_branch_name: 1.7.latest
# PR_title: "Cleanup main after cutting new 1.7.latest branch"
# PR_body: "All adapter PRs will fail CI until the dbt-core PR has been merged due to release version conflicts."
# secrets:
# FISHTOWN_BOT_PAT: ${{ secrets.FISHTOWN_BOT_PAT }}
# TODOs
# add note to eventually commit changes directly and bypass checks - same as release - when we move to this model run test action after merge
name: Cut new release branch
run-name: "Cutting New Branch: ${{ inputs.new_branch_name }}"
on:
workflow_dispatch:
inputs:
version_to_bump_main:
description: 'The alpha version main should bump to (ex. 1.6.0a1)'
required: true
new_branch_name:
description: 'The full name of the new branch (ex. 1.5.latest)'
description: "The full name of the new branch (ex. 1.5.latest)"
required: true
type: string
defaults:
run:
@@ -27,15 +46,347 @@ defaults:
permissions:
contents: write
pull-requests: write
env:
PYTHON_TARGET_VERSION: "3.10"
PR_TITLE: "Cleanup main after cutting new ${{ inputs.new_branch_name }} branch"
PR_BODY: "All adapter PRs will fail CI until the dbt-core PR has been merged due to release version conflicts."
jobs:
cut_branch:
name: "Cut branch and clean up main for dbt-core"
uses: dbt-labs/actions/.github/workflows/cut-release-branch.yml@main
with:
version_to_bump_main: ${{ inputs.version_to_bump_main }}
new_branch_name: ${{ inputs.new_branch_name }}
PR_title: "Cleanup main after cutting new ${{ inputs.new_branch_name }} branch"
PR_body: "All adapter PRs will fail CI until the dbt-core PR has been merged due to release version conflicts."
secrets:
FISHTOWN_BOT_PAT: ${{ secrets.FISHTOWN_BOT_PAT }}
prep_work:
name: "Prep Work"
runs-on: ubuntu-latest
steps:
- name: "[DEBUG] Print Inputs"
run: |
echo "new_branch_name: ${{ inputs.new_branch_name }}"
echo "PR_title: ${{ env.PR_TITLE }}"
echo "PR_body: ${{ env.PR_BODY }}"
create_temp_branch:
name: "Create Temp branch off main"
runs-on: ubuntu-latest
outputs:
temp_branch_name: ${{ steps.variables.outputs.BRANCH_NAME }}
steps:
- name: "Set Branch Value"
id: variables
run: |
echo "BRANCH_NAME=cutting_release_branch/main_cleanup_$GITHUB_RUN_ID" >> $GITHUB_OUTPUT
- name: "Checkout ${{ github.repository }}"
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # actions/checkout@v4
with:
ref: "main"
token: ${{ secrets.FISHTOWN_BOT_PAT }}
- name: "Create PR Branch"
run: |
user="Github Build Bot"
email="buildbot@fishtownanalytics.com"
git config user.name "$user"
git config user.email "$email"
git checkout -b ${{ steps.variables.outputs.BRANCH_NAME }}
git push --set-upstream origin ${{ steps.variables.outputs.BRANCH_NAME }}
- name: "[Notification] Temp branch created"
run: |
message="Temp branch ${{ steps.variables.outputs.BRANCH_NAME }} created"
echo "::notice title="Temporary branch created": $title::$message"
cleanup_changelog:
name: "Clean Up Changelog"
needs: ["create_temp_branch"]
runs-on: ubuntu-latest
outputs:
next-version: ${{ steps.semver-current.outputs.next-minor-alpha-version }}
steps:
- name: "Checkout ${{ github.repository }}"
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # actions/checkout@v4
with:
ref: ${{ needs.create_temp_branch.outputs.temp_branch_name }}
token: ${{ secrets.FISHTOWN_BOT_PAT }}
- name: "Add Homebrew To PATH"
run: |
echo "/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin" >> $GITHUB_PATH
- name: "Install Homebrew Packages"
run: |
brew install pre-commit
brew tap miniscruff/changie https://github.com/miniscruff/changie
brew install changie
- name: "Check Current Version In Code"
id: determine_version
run: |
current_version=$(grep '^version = ' core/pyproject.toml | sed 's/version = "\(.*\)"/\1/')
echo "current_version=$current_version" >> $GITHUB_OUTPUT
- name: "[Notification] Check Current Version In Code"
run: |
message="The current version is ${{ steps.determine_version.outputs.current_version }}"
echo "::notice title="Version Bump Check": $title::$message"
- name: "Parse Current Version Into Parts for Changelog Directories"
id: semver-current
uses: dbt-labs/actions/parse-semver@main
with:
version: ${{ steps.determine_version.outputs.current_version }}
- name: "[Notification] Next Alpha Version"
run: |
message="The next alpha version is ${{ steps.semver-current.outputs.next-minor-alpha-version }}"
echo "::notice title="Version Bump Check": $title::$message"
- name: "Delete Unreleased Changelog YAMLs"
# removal fails if no files exist. OK to continue since we're just cleaning up the files.
continue-on-error: true
run: |
rm .changes/unreleased/*.yaml || true
- name: "Delete Pre Release Changelogs and YAMLs"
# removal fails if no files exist. OK to continue since we're just cleaning up the files.
continue-on-error: true
run: |
rm .changes/${{ steps.semver-current.outputs.base-version }}/*.yaml || true
rm .changes/${{ steps.semver-current.outputs.major }}.${{ steps.semver-current.outputs.minor }}.*.md || true
- name: "Cleanup CHANGELOG.md"
run: |
changie merge
- name: "Commit Changelog Cleanup to Branch"
run: |
user="Github Build Bot"
email="buildbot@fishtownanalytics.com"
git config user.name "$user"
git config user.email "$email"
git status
git add .
git commit -m "Clean up changelog on main"
git push
- name: "[Notification] Changelog cleaned up"
run: |
message="Changelog on ${{ needs.create_temp_branch.outputs.temp_branch_name }} cleaned up"
echo "::notice title="Changelog cleaned up": $title::$message"
bump_version:
name: "Bump to next minor version"
needs: ["cleanup_changelog", "create_temp_branch"]
runs-on: ubuntu-latest
steps:
- name: "Checkout ${{ github.repository }}"
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # actions/checkout@v4
with:
ref: ${{ needs.create_temp_branch.outputs.temp_branch_name }}
token: ${{ secrets.FISHTOWN_BOT_PAT }}
- name: "Set up Python - ${{ env.PYTHON_TARGET_VERSION }}"
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # actions/setup-python@v5
with:
python-version: "${{ env.PYTHON_TARGET_VERSION }}"
- name: "Install Spark Dependencies"
if: ${{ contains(github.repository, 'dbt-labs/dbt-spark') }}
run: |
sudo apt-get update
sudo apt-get install libsasl2-dev
- name: "Install Python Dependencies"
run: |
python -m venv env
source env/bin/activate
python -m pip install --upgrade pip
- name: "Bump Version To ${{ needs.cleanup_changelog.outputs.next-version }}"
run: |
source env/bin/activate
python -m pip install -r dev-requirements.txt
env/bin/bumpversion --allow-dirty --new-version ${{ needs.cleanup_changelog.outputs.next-version }} major
git status
- name: "Commit Version Bump to Branch"
run: |
user="Github Build Bot"
email="buildbot@fishtownanalytics.com"
git config user.name "$user"
git config user.email "$email"
git status
git add .
git commit -m "Bumping version to ${{ needs.cleanup_changelog.outputs.next-version }}"
git push
- name: "[Notification] Version Bump completed"
run: |
message="Version on ${{ needs.create_temp_branch.outputs.temp_branch_name }} bumped to ${{ needs.cleanup_changelog.outputs.next-version }}"
echo "::notice title="Version Bump Completed": $title::$message"
cleanup:
name: "Cleanup Code Quality"
needs: ["create_temp_branch", "bump_version"]
runs-on: ubuntu-latest
steps:
- name: "Checkout ${{ github.repository }}"
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # actions/checkout@v4
with:
ref: ${{ needs.create_temp_branch.outputs.temp_branch_name }}
token: ${{ secrets.FISHTOWN_BOT_PAT }}
- name: "Add Homebrew To PATH"
run: |
echo "/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin" >> $GITHUB_PATH
- name: "brew install pre-commit"
run: |
brew install pre-commit
# this step will fail on whitespace errors but also correct them
- name: "Cleanup - Remove Trailing Whitespace Via Pre-commit"
continue-on-error: true
run: |
pre-commit run trailing-whitespace --files .bumpversion.cfg CHANGELOG.md .changes/* || true
# this step will fail on newline errors but also correct them
- name: "Cleanup - Remove Extra Newlines Via Pre-commit"
continue-on-error: true
run: |
pre-commit run end-of-file-fixer --files .bumpversion.cfg CHANGELOG.md .changes/* || true
- name: "Commit Version Bump to Branch"
run: |
user="Github Build Bot"
email="buildbot@fishtownanalytics.com"
git config user.name "$user"
git config user.email "$email"
git status
git add .
git commit -m "Code quality cleanup"
git push
open_pr:
name: "Open PR Against main"
needs: ["cleanup_changelog", "create_temp_branch", "cleanup"]
runs-on: ubuntu-latest
outputs:
pr_number: ${{ steps.create_pr.outputs.pull-request-number }}
steps:
- name: "Checkout ${{ github.repository }}"
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # actions/checkout@v4
with:
ref: ${{ needs.create_temp_branch.outputs.temp_branch_name }}
token: ${{ secrets.FISHTOWN_BOT_PAT }}
- name: "Determine PR Title"
id: pr_title
run: |
echo "pr_title=${{ env.PR_TITLE }}" >> $GITHUB_OUTPUT
if [${{ env.PR_TITLE }} == ""]; then
echo "pr_title='Clean up changelogs and bump to version ${{ needs.cleanup_changelog.outputs.next-version }}'" >> $GITHUB_OUTPUT
fi
- name: "Determine PR Body"
id: pr_body
run: |
echo "pr_body=${{ env.PR_BODY }}" >> $GITHUB_OUTPUT
if [${{ env.PR_BODY }} == ""]; then
echo "pr_body='Clean up changelogs and bump to version ${{ needs.cleanup_changelog.outputs.next-version }}'" >> $GITHUB_OUTPUT
fi
- name: "Add Branch Details"
id: pr_body_branch
run: |
branch_details="The workflow that generated this PR also created a new branch: ${{ inputs.new_branch_name }}"
full_body="${{ steps.pr_body.outputs.pr_body }} $branch_details"
echo "pr_full_body=$full_body" >> $GITHUB_OUTPUT
- name: "Open Pull Request"
id: create_pr
run: |
pr_url=$(gh pr create -B main -H ${{ needs.create_temp_branch.outputs.temp_branch_name }} -l "Skip Changelog" -t "${{ steps.pr_title.outputs.pr_title }}" -b "${{ steps.pr_body_branch.outputs.pr_full_body }}")
echo "pr_url=$pr_url" >> $GITHUB_OUTPUT
env:
GH_TOKEN: ${{ secrets.FISHTOWN_BOT_PAT }}
- name: "[Notification] Pull Request Opened"
run: |
message="PR opened at ${{ steps.create_pr.outputs.pr_url }}"
echo "::notice title="Pull Request Opened": $title::$message"
cut_new_branch:
# don't cut the new branch until we're done opening the PR against main
name: "Cut New Branch ${{ inputs.new_branch_name }}"
needs: [open_pr]
runs-on: ubuntu-latest
steps:
- name: "Checkout ${{ github.repository }}"
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # actions/checkout@v4
with:
token: ${{ secrets.FISHTOWN_BOT_PAT }}
fetch-depth: 0
- name: "Ensure New Branch Does Not Exist"
id: check_new_branch
run: |
title="Check New Branch Existence"
if git show-ref --quiet ${{ inputs.new_branch_name }}; then
message="Branch ${{ inputs.new_branch_name }} already exists. Exiting."
echo "::error $title::$message"
exit 1
fi
- name: "Create New Release Branch"
run: |
git checkout -b ${{ inputs.new_branch_name }}
- name: "Push up New Branch"
run: |
#Data for commit
user="Github Build Bot"
email="buildbot@fishtownanalytics.com"
git config user.name "$user"
git config user.email "$email"
git push --set-upstream origin ${{ inputs.new_branch_name }}
- name: "[Notification] New branch created"
run: |
message="New branch ${{ inputs.new_branch_name }} created"
echo "::notice title="New branch created": $title::$message"
- name: "Bump dependencies via script"
# This bumps the dependency on dbt-core in the adapters
if: ${{ !contains(github.repository, 'dbt-core') }}
run: |
echo ${{ github.repository }}
echo "running update_dependencies script"
bash ${GITHUB_WORKSPACE}/.github/scripts/update_dependencies.sh ${{ inputs.new_branch_name }}
commit_message="bumping .latest branch variable in update_dependencies.sh to ${{ inputs.new_branch_name }}"
git status
git add .
git commit -m "$commit_message"
git push
- name: "Bump env variable via script"
# bumps the RELEASE_BRANCH variable in nightly-release.yml in adapters
if: ${{ !contains(github.repository, 'dbt-core') }}
run: |
file="./.github/scripts/update_release_branch.sh"
if test -f "$file"; then
echo ${{ github.repository }}
echo "running some script yet to be written now"
bash $file ${{ inputs.new_branch_name }}
commit_message="updating env variable to ${{ inputs.new_branch_name }} in nightly-release.yml"
git status
git add .
git commit -m "$commit_message"
git push
else
echo "no $file seen skipping step"
fi

View File

@@ -5,6 +5,36 @@
- "Breaking changes" listed under a version may require action from end users or external maintainers when upgrading to that version.
- Do not edit this file directly. This file is auto-generated using [changie](https://github.com/miniscruff/changie). For details on how to document a change, see [the contributing guide](https://github.com/dbt-labs/dbt-core/blob/main/CONTRIBUTING.md#adding-changelog-entry)
## dbt-core 1.11.0-rc1 - November 18, 2025
### Features
- Allow for defining funciton arguments with default values ([#12044](https://github.com/dbt-labs/dbt-core/issues/12044))
### Fixes
- Fix parse error when build_after.count set to 0 ([#12136](https://github.com/dbt-labs/dbt-core/issues/12136))
- Stop compiling python udfs like python models ([#12153](https://github.com/dbt-labs/dbt-core/issues/12153))
- For metric names, fix bug allowing hyphens (not allowed in metricflow already), make validation throw ValidationErrors (not ParsingErrors), and add tests. ([#n/a](https://github.com/dbt-labs/dbt-core/issues/n/a))
- Include macros in unit test parsing ([#10157](https://github.com/dbt-labs/dbt-core/issues/10157))
### Under the Hood
- add dbt/jsonschemas to manifest.in ([#12126](https://github.com/dbt-labs/dbt-core/issues/12126))
- Move from setup.py to pyproject.toml ([#5696](https://github.com/dbt-labs/dbt-core/issues/5696))
- Fixes issue where config isn't propagated to metric from measure when set as create_metric:True ([#None](https://github.com/dbt-labs/dbt-core/issues/None))
- Support DBT_ENGINE prefix for record-mode env vars ([#12149](https://github.com/dbt-labs/dbt-core/issues/12149))
### Dependencies
- Drop support for python 3.9 ([#12118](https://github.com/dbt-labs/dbt-core/issues/12118))
### Contributors
- [@WilliamDee](https://github.com/WilliamDee) ([#None](https://github.com/dbt-labs/dbt-core/issues/None))
- [@nathanskone](https://github.com/nathanskone) ([#10157](https://github.com/dbt-labs/dbt-core/issues/10157))
- [@theyostalservice](https://github.com/theyostalservice) ([#n/a](https://github.com/dbt-labs/dbt-core/issues/n/a))
## dbt-core 1.11.0-b4 - October 28, 2025
### Features
@@ -29,7 +59,6 @@
- [@12030](https://github.com/12030) ([#QMalcolm](https://github.com/dbt-labs/dbt-core/issues/QMalcolm))
- [@WilliamDee](https://github.com/WilliamDee) ([#None](https://github.com/dbt-labs/dbt-core/issues/None))
## dbt-core 1.11.0-b3 - October 07, 2025
### Features

View File

@@ -1,5 +1,5 @@
from dataclasses import dataclass, field
from typing import List, Literal, Optional
from typing import Any, List, Literal, Optional
from dbt.artifacts.resources.types import FunctionType, FunctionVolatility, NodeType
from dbt.artifacts.resources.v1.components import CompiledResource
@@ -32,6 +32,7 @@ class FunctionArgument(dbtClassMixin):
name: str
data_type: str
description: Optional[str] = None
default_value: Optional[Any] = None
@dataclass

View File

@@ -1905,6 +1905,21 @@ def generate_parser_model_context(
return ctx.to_dict()
def generate_parser_unit_test_context(
unit_test: UnitTestNode, config: RuntimeConfig, manifest: Manifest
) -> Dict[str, Any]:
context_config = ContextConfig(
config,
unit_test.fqn,
NodeType.Unit,
config.project_name,
)
ctx = UnitTestContext(unit_test, config, manifest, ParseProvider(), context_config)
return ctx.to_dict()
def generate_generate_name_macro_context(
macro: Macro,
config: RuntimeConfig,

View File

@@ -525,6 +525,7 @@ class ManifestLoader:
self.check_for_microbatch_deprecations()
self.check_forcing_batch_concurrency()
self.check_microbatch_model_has_a_filtered_input()
self.check_function_default_arguments_ordering()
return self.manifest
@@ -1547,6 +1548,17 @@ class ManifestLoader:
if not has_input_with_event_time_config:
fire_event(MicrobatchModelNoEventTimeInputs(model_name=node.name))
def check_function_default_arguments_ordering(self):
for function in self.manifest.functions.values():
found_default_value = False
for argument in function.arguments:
if not found_default_value and argument.default_value is not None:
found_default_value = True
elif found_default_value and argument.default_value is None:
raise dbt.exceptions.ParsingError(
f"Non-defaulted argument '{argument.name}' of function '{function.name}' comes after a defaulted argument. Non-defaulted arguments cannot come after defaulted arguments. "
)
def write_perf_info(self, target_path: str):
path = os.path.join(target_path, PERF_INFO_FILE_NAME)
write_file(path, json.dumps(self._perf_info, cls=dbt.utils.JSONEncoder, indent=4))

View File

@@ -10,7 +10,7 @@ from dbt import utils
from dbt.artifacts.resources import ModelConfig, UnitTestConfig, UnitTestFormat
from dbt.config import RuntimeConfig
from dbt.context.context_config import ContextConfig
from dbt.context.providers import generate_parse_exposure, get_rendered
from dbt.context.providers import generate_parser_unit_test_context, get_rendered
from dbt.contracts.files import FileHash, SchemaSourceFile
from dbt.contracts.graph.manifest import Manifest
from dbt.contracts.graph.model_config import UnitTestNodeConfig
@@ -100,12 +100,7 @@ class UnitTestManifestLoader:
overrides=test_case.overrides,
)
ctx = generate_parse_exposure(
unit_test_node, # type: ignore
self.root_project,
self.manifest,
test_case.package_name,
)
ctx = generate_parser_unit_test_context(unit_test_node, self.root_project, self.manifest)
get_rendered(unit_test_node.raw_code, ctx, unit_test_node, capture_macros=True)
# unit_test_node now has a populated refs/sources

View File

@@ -27,7 +27,7 @@ include = [
[project]
name = "dbt-core"
version = "1.11.0b4"
version = "1.11.0rc1"
description = "With dbt, data analysts and engineers can build analytics the way engineers build applications."
readme = "README.md"
requires-python = ">=3.10"

View File

@@ -6,6 +6,7 @@ import pytest
from dbt.artifacts.resources import FunctionReturns
from dbt.artifacts.resources.types import FunctionType, FunctionVolatility
from dbt.contracts.graph.nodes import FunctionNode
from dbt.exceptions import ParsingError
from dbt.tests.util import run_dbt, write_file
double_it_sql = """
@@ -149,6 +150,42 @@ scalar_function_python_macro = """
{% endmacro %}
"""
sum_2_values_sql = """
SELECT val1 + val2 as sum_2_values
"""
sum_2_values_yml = """
functions:
- name: sum_2_values
description: Add two values together
arguments:
- name: val1
data_type: integer
description: The first value
- name: val2
data_type: integer
description: The second value
default_value: 0
returns:
data_type: integer
"""
sum_2_values_bad_default_arg_order_yml = """
functions:
- name: sum_2_values
description: Add two values together
arguments:
- name: val1
data_type: integer
description: The first value
default_value: 0
- name: val2
data_type: integer
description: The second value
returns:
data_type: integer
"""
class TestBasicSQLUDF(BasicUDFSetup):
def test_basic_parsing(self, project):
@@ -449,3 +486,38 @@ class TestPythonFunctionWithJinjaHasCorrectCompiledCode:
node = result.results[0].node
assert isinstance(node, FunctionNode)
assert node.compiled_code == "def entry(value):\n \n return value * 2\n "
class TestDefaultArgumentsBasic:
@pytest.fixture(scope="class")
def functions(self) -> Dict[str, str]:
return {
"sum_2_values.py": sum_2_values_sql,
"sum_2_values.yml": sum_2_values_yml,
}
def test_udfs(self, project):
manifest = run_dbt(["parse"])
assert len(manifest.functions) == 1
function_node = manifest.functions["function.test.sum_2_values"]
assert isinstance(function_node, FunctionNode)
assert len(function_node.arguments) == 2
assert function_node.arguments[0].default_value is None
assert function_node.arguments[1].default_value == 0
class TestDefaultArgumentsMustComeLast:
@pytest.fixture(scope="class")
def functions(self) -> Dict[str, str]:
return {
"sum_2_values.py": sum_2_values_sql,
"sum_2_values.yml": sum_2_values_bad_default_arg_order_yml,
}
def test_udfs(self, project):
with pytest.raises(ParsingError) as excinfo:
run_dbt(["parse"])
assert (
"Non-defaulted argument 'val2' of function 'sum_2_values' comes after a defaulted argument. Non-defaulted arguments cannot come after defaulted arguments. "
in str(excinfo.value)
)

View File

@@ -0,0 +1,75 @@
import pytest
from dbt.tests.util import run_dbt
my_model_without_composition_sql = """
{{ config(materialized='table') }}
{% set one = macro_one() %}
{% set two = macro_two() %}
select 1 as id
"""
my_model_with_composition_sql = """
{{ config(materialized='table') }}
{% set one = macro_one() %}
{% set two = macro_two() %}
{% set one_plus_two = one + two %}
select 1 as id
"""
my_macro_sql = """
{% macro macro_one() -%}
{{ return(1) }}
{%- endmacro %}
{% macro macro_two() -%}
{{ return(2) }}
{%- endmacro %}
"""
my_unit_test_yml = """
unit_tests:
- name: my_unit_test
model: my_model
given: []
expect:
rows:
- {id: 1}
"""
class TestMacroWithoutComposition:
@pytest.fixture(scope="class")
def models(self):
return {
"my_model.sql": my_model_without_composition_sql,
"my_unit_test.yml": my_unit_test_yml,
}
@pytest.fixture(scope="class")
def macros(self):
return {"my_macros.sql": my_macro_sql}
def test_macro_in_unit_test(self, project):
# Test that a model without macro composition properly resolves macro names in unit tests
run_dbt(["test"])
class TestMacroComposition:
@pytest.fixture(scope="class")
def models(self):
return {
"my_model.sql": my_model_with_composition_sql,
"my_unit_test.yml": my_unit_test_yml,
}
@pytest.fixture(scope="class")
def macros(self):
return {"my_macros.sql": my_macro_sql}
def test_macro_composition_in_unit_test(self, project):
# Verify model works fine outside of unit testing
results = run_dbt(["run"])
assert len(results) == 1
# Test that a model with macro composition properly resolves macro names in unit tests
run_dbt(["test"])

View File

@@ -0,0 +1,87 @@
import pytest
from dbt.tests.util import run_dbt, run_dbt_and_capture
dbt_project_yml = """
vars:
columns_list_one:
- column_a
- column_b
columns_list_two:
- column_c
"""
my_model_one_variable_sql = """
{{ config(materialized='table') }}
-- {{ get_columns(include=var('columns_list_one'))}}
select 1 as id
"""
my_model_two_variables_sql = """
{{ config(materialized='table') }}
-- {{ get_columns(include=var('columns_list_one') + var('columns_list_two'))}}
select 1 as id
"""
my_macro_sql = """
{%- macro get_columns(include=[]) -%}
{%- for col in include -%}
{{ col }}{% if not loop.last %}, {% endif %}
{%- endfor -%}
{%- endmacro -%}
"""
my_unit_test_yml = """
unit_tests:
- name: my_unit_test
model: my_model
given: []
expect:
rows:
- {id: 1}
"""
class TestUnitTestOneVariables:
@pytest.fixture(scope="class")
def project_config_update(self):
return dbt_project_yml
@pytest.fixture(scope="class")
def models(self):
return {
"my_model.sql": my_model_one_variable_sql,
"my_unit_test.yml": my_unit_test_yml,
}
@pytest.fixture(scope="class")
def macros(self):
return {"my_macros.sql": my_macro_sql}
def test_one_variable_as_input_to_macro(self, project):
run_dbt_and_capture(["test"], expect_pass=True)
class TestUnitTestTwoVariables:
@pytest.fixture(scope="class")
def project_config_update(self):
return dbt_project_yml
@pytest.fixture(scope="class")
def models(self):
return {
"my_model.sql": my_model_two_variables_sql,
"my_unit_test.yml": my_unit_test_yml,
}
@pytest.fixture(scope="class")
def macros(self):
return {"my_macros.sql": my_macro_sql}
def test_two_variables_as_input_to_macro(self, project):
# Verify model works fine outside of unit testing
results = run_dbt(["run"])
assert len(results) == 1
run_dbt(["test"])