Compare commits

...

4 Commits

Author SHA1 Message Date
Github Build Bot
ee6a69a7c3 Bumping version to 1.11.0rc1 and generate changelog 2025-11-18 18:49:22 +00:00
github-actions[bot]
0d59a3d5a4 Fix 10157: macro composition in unit tests (#12168) (#12182)
(cherry picked from commit f6e0793d00)

Co-authored-by: Michelle Ark <MichelleArk@users.noreply.github.com>
2025-11-18 13:45:13 -05:00
github-actions[bot]
b52eb6f8e7 Support default arguments for udfs (#12175) (#12181)
* Add tests to check parsing of function argument default values

* Begin allowing the specification of `default_value` on function arguments

* Validate that non-default function arguments don't come _after_ default function arguments

* Add changie doc

(cherry picked from commit 811e4ee955)

Co-authored-by: Quigley Malcolm <QMalcolm@users.noreply.github.com>
2025-11-18 12:33:53 -06:00
Emily Rockman
8666c83f26 move full workflow into core since no one else uses it 2025-11-18 09:55:58 -05:00
22 changed files with 707 additions and 30 deletions

View File

@@ -1,5 +1,5 @@
[bumpversion] [bumpversion]
current_version = 1.11.0b4 current_version = 1.11.0rc1
parse = (?P<major>[\d]+) # major version number parse = (?P<major>[\d]+) # major version number
\.(?P<minor>[\d]+) # minor version number \.(?P<minor>[\d]+) # minor version number
\.(?P<patch>[\d]+) # patch version number \.(?P<patch>[\d]+) # patch version number

28
.changes/1.11.0-rc1.md Normal file
View File

@@ -0,0 +1,28 @@
## dbt-core 1.11.0-rc1 - November 18, 2025
### Features
- Allow for defining funciton arguments with default values ([#12044](https://github.com/dbt-labs/dbt-core/issues/12044))
### Fixes
- Fix parse error when build_after.count set to 0 ([#12136](https://github.com/dbt-labs/dbt-core/issues/12136))
- Stop compiling python udfs like python models ([#12153](https://github.com/dbt-labs/dbt-core/issues/12153))
- For metric names, fix bug allowing hyphens (not allowed in metricflow already), make validation throw ValidationErrors (not ParsingErrors), and add tests. ([#n/a](https://github.com/dbt-labs/dbt-core/issues/n/a))
- Include macros in unit test parsing ([#10157](https://github.com/dbt-labs/dbt-core/issues/10157))
### Under the Hood
- add dbt/jsonschemas to manifest.in ([#12126](https://github.com/dbt-labs/dbt-core/issues/12126))
- Move from setup.py to pyproject.toml ([#5696](https://github.com/dbt-labs/dbt-core/issues/5696))
- Fixes issue where config isn't propagated to metric from measure when set as create_metric:True ([#None](https://github.com/dbt-labs/dbt-core/issues/None))
- Support DBT_ENGINE prefix for record-mode env vars ([#12149](https://github.com/dbt-labs/dbt-core/issues/12149))
### Dependencies
- Drop support for python 3.9 ([#12118](https://github.com/dbt-labs/dbt-core/issues/12118))
### Contributors
- [@WilliamDee](https://github.com/WilliamDee) ([#None](https://github.com/dbt-labs/dbt-core/issues/None))
- [@nathanskone](https://github.com/nathanskone) ([#10157](https://github.com/dbt-labs/dbt-core/issues/10157))
- [@theyostalservice](https://github.com/theyostalservice) ([#n/a](https://github.com/dbt-labs/dbt-core/issues/n/a))

View File

@@ -0,0 +1,6 @@
kind: Features
body: Allow for defining funciton arguments with default values
time: 2025-11-17T14:10:53.860178-06:00
custom:
Author: QMalcolm
Issue: "12044"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Include macros in unit test parsing
time: 2025-11-17T14:06:49.518566-05:00
custom:
Author: michelleark nathanskone
Issue: "10157"

View File

@@ -1,25 +1,44 @@
# **what?** # **what?**
# Cuts a new `*.latest` branch # Cuts the `*.latest` branch, bumps dependencies on it, cleans up all files in `.changes/unreleased`
# Also cleans up all files in `.changes/unreleased` and `.changes/previous verion on # and `.changes/previous verion on main and bumps main to the input version.
# `main` and bumps `main` to the input version.
# **why?** # **why?**
# Generally reduces the workload of engineers and reduces error. Allow automation. # Clean up the main branch after a release branch is cut and automate cutting the release branch.
# Generally reduces the workload of engineers and reducing error.
# **when?** # **when?**
# This will run when called manually. # This will run when called manually or when triggered in another workflow.
# Example Usage including required permissions: TODO: update once finalized
# permissions:
# contents: read
# pull-requests: write
#
# name: Cut Release Branch
# jobs:
# changelog:
# uses: dbt-labs/actions/.github/workflows/cut-release-branch.yml@main
# with:
# new_branch_name: 1.7.latest
# PR_title: "Cleanup main after cutting new 1.7.latest branch"
# PR_body: "All adapter PRs will fail CI until the dbt-core PR has been merged due to release version conflicts."
# secrets:
# FISHTOWN_BOT_PAT: ${{ secrets.FISHTOWN_BOT_PAT }}
# TODOs
# add note to eventually commit changes directly and bypass checks - same as release - when we move to this model run test action after merge
name: Cut new release branch name: Cut new release branch
run-name: "Cutting New Branch: ${{ inputs.new_branch_name }}"
on: on:
workflow_dispatch: workflow_dispatch:
inputs: inputs:
version_to_bump_main:
description: 'The alpha version main should bump to (ex. 1.6.0a1)'
required: true
new_branch_name: new_branch_name:
description: 'The full name of the new branch (ex. 1.5.latest)' description: "The full name of the new branch (ex. 1.5.latest)"
required: true required: true
type: string
defaults: defaults:
run: run:
@@ -27,15 +46,347 @@ defaults:
permissions: permissions:
contents: write contents: write
pull-requests: write
env:
PYTHON_TARGET_VERSION: "3.10"
PR_TITLE: "Cleanup main after cutting new ${{ inputs.new_branch_name }} branch"
PR_BODY: "All adapter PRs will fail CI until the dbt-core PR has been merged due to release version conflicts."
jobs: jobs:
cut_branch: prep_work:
name: "Cut branch and clean up main for dbt-core" name: "Prep Work"
uses: dbt-labs/actions/.github/workflows/cut-release-branch.yml@main runs-on: ubuntu-latest
with: steps:
version_to_bump_main: ${{ inputs.version_to_bump_main }} - name: "[DEBUG] Print Inputs"
new_branch_name: ${{ inputs.new_branch_name }} run: |
PR_title: "Cleanup main after cutting new ${{ inputs.new_branch_name }} branch" echo "new_branch_name: ${{ inputs.new_branch_name }}"
PR_body: "All adapter PRs will fail CI until the dbt-core PR has been merged due to release version conflicts." echo "PR_title: ${{ env.PR_TITLE }}"
secrets: echo "PR_body: ${{ env.PR_BODY }}"
FISHTOWN_BOT_PAT: ${{ secrets.FISHTOWN_BOT_PAT }}
create_temp_branch:
name: "Create Temp branch off main"
runs-on: ubuntu-latest
outputs:
temp_branch_name: ${{ steps.variables.outputs.BRANCH_NAME }}
steps:
- name: "Set Branch Value"
id: variables
run: |
echo "BRANCH_NAME=cutting_release_branch/main_cleanup_$GITHUB_RUN_ID" >> $GITHUB_OUTPUT
- name: "Checkout ${{ github.repository }}"
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # actions/checkout@v4
with:
ref: "main"
token: ${{ secrets.FISHTOWN_BOT_PAT }}
- name: "Create PR Branch"
run: |
user="Github Build Bot"
email="buildbot@fishtownanalytics.com"
git config user.name "$user"
git config user.email "$email"
git checkout -b ${{ steps.variables.outputs.BRANCH_NAME }}
git push --set-upstream origin ${{ steps.variables.outputs.BRANCH_NAME }}
- name: "[Notification] Temp branch created"
run: |
message="Temp branch ${{ steps.variables.outputs.BRANCH_NAME }} created"
echo "::notice title="Temporary branch created": $title::$message"
cleanup_changelog:
name: "Clean Up Changelog"
needs: ["create_temp_branch"]
runs-on: ubuntu-latest
outputs:
next-version: ${{ steps.semver-current.outputs.next-minor-alpha-version }}
steps:
- name: "Checkout ${{ github.repository }}"
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # actions/checkout@v4
with:
ref: ${{ needs.create_temp_branch.outputs.temp_branch_name }}
token: ${{ secrets.FISHTOWN_BOT_PAT }}
- name: "Add Homebrew To PATH"
run: |
echo "/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin" >> $GITHUB_PATH
- name: "Install Homebrew Packages"
run: |
brew install pre-commit
brew tap miniscruff/changie https://github.com/miniscruff/changie
brew install changie
- name: "Check Current Version In Code"
id: determine_version
run: |
current_version=$(grep '^version = ' core/pyproject.toml | sed 's/version = "\(.*\)"/\1/')
echo "current_version=$current_version" >> $GITHUB_OUTPUT
- name: "[Notification] Check Current Version In Code"
run: |
message="The current version is ${{ steps.determine_version.outputs.current_version }}"
echo "::notice title="Version Bump Check": $title::$message"
- name: "Parse Current Version Into Parts for Changelog Directories"
id: semver-current
uses: dbt-labs/actions/parse-semver@main
with:
version: ${{ steps.determine_version.outputs.current_version }}
- name: "[Notification] Next Alpha Version"
run: |
message="The next alpha version is ${{ steps.semver-current.outputs.next-minor-alpha-version }}"
echo "::notice title="Version Bump Check": $title::$message"
- name: "Delete Unreleased Changelog YAMLs"
# removal fails if no files exist. OK to continue since we're just cleaning up the files.
continue-on-error: true
run: |
rm .changes/unreleased/*.yaml || true
- name: "Delete Pre Release Changelogs and YAMLs"
# removal fails if no files exist. OK to continue since we're just cleaning up the files.
continue-on-error: true
run: |
rm .changes/${{ steps.semver-current.outputs.base-version }}/*.yaml || true
rm .changes/${{ steps.semver-current.outputs.major }}.${{ steps.semver-current.outputs.minor }}.*.md || true
- name: "Cleanup CHANGELOG.md"
run: |
changie merge
- name: "Commit Changelog Cleanup to Branch"
run: |
user="Github Build Bot"
email="buildbot@fishtownanalytics.com"
git config user.name "$user"
git config user.email "$email"
git status
git add .
git commit -m "Clean up changelog on main"
git push
- name: "[Notification] Changelog cleaned up"
run: |
message="Changelog on ${{ needs.create_temp_branch.outputs.temp_branch_name }} cleaned up"
echo "::notice title="Changelog cleaned up": $title::$message"
bump_version:
name: "Bump to next minor version"
needs: ["cleanup_changelog", "create_temp_branch"]
runs-on: ubuntu-latest
steps:
- name: "Checkout ${{ github.repository }}"
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # actions/checkout@v4
with:
ref: ${{ needs.create_temp_branch.outputs.temp_branch_name }}
token: ${{ secrets.FISHTOWN_BOT_PAT }}
- name: "Set up Python - ${{ env.PYTHON_TARGET_VERSION }}"
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # actions/setup-python@v5
with:
python-version: "${{ env.PYTHON_TARGET_VERSION }}"
- name: "Install Spark Dependencies"
if: ${{ contains(github.repository, 'dbt-labs/dbt-spark') }}
run: |
sudo apt-get update
sudo apt-get install libsasl2-dev
- name: "Install Python Dependencies"
run: |
python -m venv env
source env/bin/activate
python -m pip install --upgrade pip
- name: "Bump Version To ${{ needs.cleanup_changelog.outputs.next-version }}"
run: |
source env/bin/activate
python -m pip install -r dev-requirements.txt
env/bin/bumpversion --allow-dirty --new-version ${{ needs.cleanup_changelog.outputs.next-version }} major
git status
- name: "Commit Version Bump to Branch"
run: |
user="Github Build Bot"
email="buildbot@fishtownanalytics.com"
git config user.name "$user"
git config user.email "$email"
git status
git add .
git commit -m "Bumping version to ${{ needs.cleanup_changelog.outputs.next-version }}"
git push
- name: "[Notification] Version Bump completed"
run: |
message="Version on ${{ needs.create_temp_branch.outputs.temp_branch_name }} bumped to ${{ needs.cleanup_changelog.outputs.next-version }}"
echo "::notice title="Version Bump Completed": $title::$message"
cleanup:
name: "Cleanup Code Quality"
needs: ["create_temp_branch", "bump_version"]
runs-on: ubuntu-latest
steps:
- name: "Checkout ${{ github.repository }}"
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # actions/checkout@v4
with:
ref: ${{ needs.create_temp_branch.outputs.temp_branch_name }}
token: ${{ secrets.FISHTOWN_BOT_PAT }}
- name: "Add Homebrew To PATH"
run: |
echo "/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin" >> $GITHUB_PATH
- name: "brew install pre-commit"
run: |
brew install pre-commit
# this step will fail on whitespace errors but also correct them
- name: "Cleanup - Remove Trailing Whitespace Via Pre-commit"
continue-on-error: true
run: |
pre-commit run trailing-whitespace --files .bumpversion.cfg CHANGELOG.md .changes/* || true
# this step will fail on newline errors but also correct them
- name: "Cleanup - Remove Extra Newlines Via Pre-commit"
continue-on-error: true
run: |
pre-commit run end-of-file-fixer --files .bumpversion.cfg CHANGELOG.md .changes/* || true
- name: "Commit Version Bump to Branch"
run: |
user="Github Build Bot"
email="buildbot@fishtownanalytics.com"
git config user.name "$user"
git config user.email "$email"
git status
git add .
git commit -m "Code quality cleanup"
git push
open_pr:
name: "Open PR Against main"
needs: ["cleanup_changelog", "create_temp_branch", "cleanup"]
runs-on: ubuntu-latest
outputs:
pr_number: ${{ steps.create_pr.outputs.pull-request-number }}
steps:
- name: "Checkout ${{ github.repository }}"
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # actions/checkout@v4
with:
ref: ${{ needs.create_temp_branch.outputs.temp_branch_name }}
token: ${{ secrets.FISHTOWN_BOT_PAT }}
- name: "Determine PR Title"
id: pr_title
run: |
echo "pr_title=${{ env.PR_TITLE }}" >> $GITHUB_OUTPUT
if [${{ env.PR_TITLE }} == ""]; then
echo "pr_title='Clean up changelogs and bump to version ${{ needs.cleanup_changelog.outputs.next-version }}'" >> $GITHUB_OUTPUT
fi
- name: "Determine PR Body"
id: pr_body
run: |
echo "pr_body=${{ env.PR_BODY }}" >> $GITHUB_OUTPUT
if [${{ env.PR_BODY }} == ""]; then
echo "pr_body='Clean up changelogs and bump to version ${{ needs.cleanup_changelog.outputs.next-version }}'" >> $GITHUB_OUTPUT
fi
- name: "Add Branch Details"
id: pr_body_branch
run: |
branch_details="The workflow that generated this PR also created a new branch: ${{ inputs.new_branch_name }}"
full_body="${{ steps.pr_body.outputs.pr_body }} $branch_details"
echo "pr_full_body=$full_body" >> $GITHUB_OUTPUT
- name: "Open Pull Request"
id: create_pr
run: |
pr_url=$(gh pr create -B main -H ${{ needs.create_temp_branch.outputs.temp_branch_name }} -l "Skip Changelog" -t "${{ steps.pr_title.outputs.pr_title }}" -b "${{ steps.pr_body_branch.outputs.pr_full_body }}")
echo "pr_url=$pr_url" >> $GITHUB_OUTPUT
env:
GH_TOKEN: ${{ secrets.FISHTOWN_BOT_PAT }}
- name: "[Notification] Pull Request Opened"
run: |
message="PR opened at ${{ steps.create_pr.outputs.pr_url }}"
echo "::notice title="Pull Request Opened": $title::$message"
cut_new_branch:
# don't cut the new branch until we're done opening the PR against main
name: "Cut New Branch ${{ inputs.new_branch_name }}"
needs: [open_pr]
runs-on: ubuntu-latest
steps:
- name: "Checkout ${{ github.repository }}"
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # actions/checkout@v4
with:
token: ${{ secrets.FISHTOWN_BOT_PAT }}
fetch-depth: 0
- name: "Ensure New Branch Does Not Exist"
id: check_new_branch
run: |
title="Check New Branch Existence"
if git show-ref --quiet ${{ inputs.new_branch_name }}; then
message="Branch ${{ inputs.new_branch_name }} already exists. Exiting."
echo "::error $title::$message"
exit 1
fi
- name: "Create New Release Branch"
run: |
git checkout -b ${{ inputs.new_branch_name }}
- name: "Push up New Branch"
run: |
#Data for commit
user="Github Build Bot"
email="buildbot@fishtownanalytics.com"
git config user.name "$user"
git config user.email "$email"
git push --set-upstream origin ${{ inputs.new_branch_name }}
- name: "[Notification] New branch created"
run: |
message="New branch ${{ inputs.new_branch_name }} created"
echo "::notice title="New branch created": $title::$message"
- name: "Bump dependencies via script"
# This bumps the dependency on dbt-core in the adapters
if: ${{ !contains(github.repository, 'dbt-core') }}
run: |
echo ${{ github.repository }}
echo "running update_dependencies script"
bash ${GITHUB_WORKSPACE}/.github/scripts/update_dependencies.sh ${{ inputs.new_branch_name }}
commit_message="bumping .latest branch variable in update_dependencies.sh to ${{ inputs.new_branch_name }}"
git status
git add .
git commit -m "$commit_message"
git push
- name: "Bump env variable via script"
# bumps the RELEASE_BRANCH variable in nightly-release.yml in adapters
if: ${{ !contains(github.repository, 'dbt-core') }}
run: |
file="./.github/scripts/update_release_branch.sh"
if test -f "$file"; then
echo ${{ github.repository }}
echo "running some script yet to be written now"
bash $file ${{ inputs.new_branch_name }}
commit_message="updating env variable to ${{ inputs.new_branch_name }} in nightly-release.yml"
git status
git add .
git commit -m "$commit_message"
git push
else
echo "no $file seen skipping step"
fi

View File

@@ -5,6 +5,36 @@
- "Breaking changes" listed under a version may require action from end users or external maintainers when upgrading to that version. - "Breaking changes" listed under a version may require action from end users or external maintainers when upgrading to that version.
- Do not edit this file directly. This file is auto-generated using [changie](https://github.com/miniscruff/changie). For details on how to document a change, see [the contributing guide](https://github.com/dbt-labs/dbt-core/blob/main/CONTRIBUTING.md#adding-changelog-entry) - Do not edit this file directly. This file is auto-generated using [changie](https://github.com/miniscruff/changie). For details on how to document a change, see [the contributing guide](https://github.com/dbt-labs/dbt-core/blob/main/CONTRIBUTING.md#adding-changelog-entry)
## dbt-core 1.11.0-rc1 - November 18, 2025
### Features
- Allow for defining funciton arguments with default values ([#12044](https://github.com/dbt-labs/dbt-core/issues/12044))
### Fixes
- Fix parse error when build_after.count set to 0 ([#12136](https://github.com/dbt-labs/dbt-core/issues/12136))
- Stop compiling python udfs like python models ([#12153](https://github.com/dbt-labs/dbt-core/issues/12153))
- For metric names, fix bug allowing hyphens (not allowed in metricflow already), make validation throw ValidationErrors (not ParsingErrors), and add tests. ([#n/a](https://github.com/dbt-labs/dbt-core/issues/n/a))
- Include macros in unit test parsing ([#10157](https://github.com/dbt-labs/dbt-core/issues/10157))
### Under the Hood
- add dbt/jsonschemas to manifest.in ([#12126](https://github.com/dbt-labs/dbt-core/issues/12126))
- Move from setup.py to pyproject.toml ([#5696](https://github.com/dbt-labs/dbt-core/issues/5696))
- Fixes issue where config isn't propagated to metric from measure when set as create_metric:True ([#None](https://github.com/dbt-labs/dbt-core/issues/None))
- Support DBT_ENGINE prefix for record-mode env vars ([#12149](https://github.com/dbt-labs/dbt-core/issues/12149))
### Dependencies
- Drop support for python 3.9 ([#12118](https://github.com/dbt-labs/dbt-core/issues/12118))
### Contributors
- [@WilliamDee](https://github.com/WilliamDee) ([#None](https://github.com/dbt-labs/dbt-core/issues/None))
- [@nathanskone](https://github.com/nathanskone) ([#10157](https://github.com/dbt-labs/dbt-core/issues/10157))
- [@theyostalservice](https://github.com/theyostalservice) ([#n/a](https://github.com/dbt-labs/dbt-core/issues/n/a))
## dbt-core 1.11.0-b4 - October 28, 2025 ## dbt-core 1.11.0-b4 - October 28, 2025
### Features ### Features
@@ -29,7 +59,6 @@
- [@12030](https://github.com/12030) ([#QMalcolm](https://github.com/dbt-labs/dbt-core/issues/QMalcolm)) - [@12030](https://github.com/12030) ([#QMalcolm](https://github.com/dbt-labs/dbt-core/issues/QMalcolm))
- [@WilliamDee](https://github.com/WilliamDee) ([#None](https://github.com/dbt-labs/dbt-core/issues/None)) - [@WilliamDee](https://github.com/WilliamDee) ([#None](https://github.com/dbt-labs/dbt-core/issues/None))
## dbt-core 1.11.0-b3 - October 07, 2025 ## dbt-core 1.11.0-b3 - October 07, 2025
### Features ### Features

View File

@@ -1,5 +1,5 @@
from dataclasses import dataclass, field from dataclasses import dataclass, field
from typing import List, Literal, Optional from typing import Any, List, Literal, Optional
from dbt.artifacts.resources.types import FunctionType, FunctionVolatility, NodeType from dbt.artifacts.resources.types import FunctionType, FunctionVolatility, NodeType
from dbt.artifacts.resources.v1.components import CompiledResource from dbt.artifacts.resources.v1.components import CompiledResource
@@ -32,6 +32,7 @@ class FunctionArgument(dbtClassMixin):
name: str name: str
data_type: str data_type: str
description: Optional[str] = None description: Optional[str] = None
default_value: Optional[Any] = None
@dataclass @dataclass

View File

@@ -1905,6 +1905,21 @@ def generate_parser_model_context(
return ctx.to_dict() return ctx.to_dict()
def generate_parser_unit_test_context(
unit_test: UnitTestNode, config: RuntimeConfig, manifest: Manifest
) -> Dict[str, Any]:
context_config = ContextConfig(
config,
unit_test.fqn,
NodeType.Unit,
config.project_name,
)
ctx = UnitTestContext(unit_test, config, manifest, ParseProvider(), context_config)
return ctx.to_dict()
def generate_generate_name_macro_context( def generate_generate_name_macro_context(
macro: Macro, macro: Macro,
config: RuntimeConfig, config: RuntimeConfig,

View File

@@ -525,6 +525,7 @@ class ManifestLoader:
self.check_for_microbatch_deprecations() self.check_for_microbatch_deprecations()
self.check_forcing_batch_concurrency() self.check_forcing_batch_concurrency()
self.check_microbatch_model_has_a_filtered_input() self.check_microbatch_model_has_a_filtered_input()
self.check_function_default_arguments_ordering()
return self.manifest return self.manifest
@@ -1547,6 +1548,17 @@ class ManifestLoader:
if not has_input_with_event_time_config: if not has_input_with_event_time_config:
fire_event(MicrobatchModelNoEventTimeInputs(model_name=node.name)) fire_event(MicrobatchModelNoEventTimeInputs(model_name=node.name))
def check_function_default_arguments_ordering(self):
for function in self.manifest.functions.values():
found_default_value = False
for argument in function.arguments:
if not found_default_value and argument.default_value is not None:
found_default_value = True
elif found_default_value and argument.default_value is None:
raise dbt.exceptions.ParsingError(
f"Non-defaulted argument '{argument.name}' of function '{function.name}' comes after a defaulted argument. Non-defaulted arguments cannot come after defaulted arguments. "
)
def write_perf_info(self, target_path: str): def write_perf_info(self, target_path: str):
path = os.path.join(target_path, PERF_INFO_FILE_NAME) path = os.path.join(target_path, PERF_INFO_FILE_NAME)
write_file(path, json.dumps(self._perf_info, cls=dbt.utils.JSONEncoder, indent=4)) write_file(path, json.dumps(self._perf_info, cls=dbt.utils.JSONEncoder, indent=4))

View File

@@ -10,7 +10,7 @@ from dbt import utils
from dbt.artifacts.resources import ModelConfig, UnitTestConfig, UnitTestFormat from dbt.artifacts.resources import ModelConfig, UnitTestConfig, UnitTestFormat
from dbt.config import RuntimeConfig from dbt.config import RuntimeConfig
from dbt.context.context_config import ContextConfig from dbt.context.context_config import ContextConfig
from dbt.context.providers import generate_parse_exposure, get_rendered from dbt.context.providers import generate_parser_unit_test_context, get_rendered
from dbt.contracts.files import FileHash, SchemaSourceFile from dbt.contracts.files import FileHash, SchemaSourceFile
from dbt.contracts.graph.manifest import Manifest from dbt.contracts.graph.manifest import Manifest
from dbt.contracts.graph.model_config import UnitTestNodeConfig from dbt.contracts.graph.model_config import UnitTestNodeConfig
@@ -100,12 +100,7 @@ class UnitTestManifestLoader:
overrides=test_case.overrides, overrides=test_case.overrides,
) )
ctx = generate_parse_exposure( ctx = generate_parser_unit_test_context(unit_test_node, self.root_project, self.manifest)
unit_test_node, # type: ignore
self.root_project,
self.manifest,
test_case.package_name,
)
get_rendered(unit_test_node.raw_code, ctx, unit_test_node, capture_macros=True) get_rendered(unit_test_node.raw_code, ctx, unit_test_node, capture_macros=True)
# unit_test_node now has a populated refs/sources # unit_test_node now has a populated refs/sources

View File

@@ -27,7 +27,7 @@ include = [
[project] [project]
name = "dbt-core" name = "dbt-core"
version = "1.11.0b4" version = "1.11.0rc1"
description = "With dbt, data analysts and engineers can build analytics the way engineers build applications." description = "With dbt, data analysts and engineers can build analytics the way engineers build applications."
readme = "README.md" readme = "README.md"
requires-python = ">=3.10" requires-python = ">=3.10"

View File

@@ -6,6 +6,7 @@ import pytest
from dbt.artifacts.resources import FunctionReturns from dbt.artifacts.resources import FunctionReturns
from dbt.artifacts.resources.types import FunctionType, FunctionVolatility from dbt.artifacts.resources.types import FunctionType, FunctionVolatility
from dbt.contracts.graph.nodes import FunctionNode from dbt.contracts.graph.nodes import FunctionNode
from dbt.exceptions import ParsingError
from dbt.tests.util import run_dbt, write_file from dbt.tests.util import run_dbt, write_file
double_it_sql = """ double_it_sql = """
@@ -149,6 +150,42 @@ scalar_function_python_macro = """
{% endmacro %} {% endmacro %}
""" """
sum_2_values_sql = """
SELECT val1 + val2 as sum_2_values
"""
sum_2_values_yml = """
functions:
- name: sum_2_values
description: Add two values together
arguments:
- name: val1
data_type: integer
description: The first value
- name: val2
data_type: integer
description: The second value
default_value: 0
returns:
data_type: integer
"""
sum_2_values_bad_default_arg_order_yml = """
functions:
- name: sum_2_values
description: Add two values together
arguments:
- name: val1
data_type: integer
description: The first value
default_value: 0
- name: val2
data_type: integer
description: The second value
returns:
data_type: integer
"""
class TestBasicSQLUDF(BasicUDFSetup): class TestBasicSQLUDF(BasicUDFSetup):
def test_basic_parsing(self, project): def test_basic_parsing(self, project):
@@ -449,3 +486,38 @@ class TestPythonFunctionWithJinjaHasCorrectCompiledCode:
node = result.results[0].node node = result.results[0].node
assert isinstance(node, FunctionNode) assert isinstance(node, FunctionNode)
assert node.compiled_code == "def entry(value):\n \n return value * 2\n " assert node.compiled_code == "def entry(value):\n \n return value * 2\n "
class TestDefaultArgumentsBasic:
@pytest.fixture(scope="class")
def functions(self) -> Dict[str, str]:
return {
"sum_2_values.py": sum_2_values_sql,
"sum_2_values.yml": sum_2_values_yml,
}
def test_udfs(self, project):
manifest = run_dbt(["parse"])
assert len(manifest.functions) == 1
function_node = manifest.functions["function.test.sum_2_values"]
assert isinstance(function_node, FunctionNode)
assert len(function_node.arguments) == 2
assert function_node.arguments[0].default_value is None
assert function_node.arguments[1].default_value == 0
class TestDefaultArgumentsMustComeLast:
@pytest.fixture(scope="class")
def functions(self) -> Dict[str, str]:
return {
"sum_2_values.py": sum_2_values_sql,
"sum_2_values.yml": sum_2_values_bad_default_arg_order_yml,
}
def test_udfs(self, project):
with pytest.raises(ParsingError) as excinfo:
run_dbt(["parse"])
assert (
"Non-defaulted argument 'val2' of function 'sum_2_values' comes after a defaulted argument. Non-defaulted arguments cannot come after defaulted arguments. "
in str(excinfo.value)
)

View File

@@ -0,0 +1,75 @@
import pytest
from dbt.tests.util import run_dbt
my_model_without_composition_sql = """
{{ config(materialized='table') }}
{% set one = macro_one() %}
{% set two = macro_two() %}
select 1 as id
"""
my_model_with_composition_sql = """
{{ config(materialized='table') }}
{% set one = macro_one() %}
{% set two = macro_two() %}
{% set one_plus_two = one + two %}
select 1 as id
"""
my_macro_sql = """
{% macro macro_one() -%}
{{ return(1) }}
{%- endmacro %}
{% macro macro_two() -%}
{{ return(2) }}
{%- endmacro %}
"""
my_unit_test_yml = """
unit_tests:
- name: my_unit_test
model: my_model
given: []
expect:
rows:
- {id: 1}
"""
class TestMacroWithoutComposition:
@pytest.fixture(scope="class")
def models(self):
return {
"my_model.sql": my_model_without_composition_sql,
"my_unit_test.yml": my_unit_test_yml,
}
@pytest.fixture(scope="class")
def macros(self):
return {"my_macros.sql": my_macro_sql}
def test_macro_in_unit_test(self, project):
# Test that a model without macro composition properly resolves macro names in unit tests
run_dbt(["test"])
class TestMacroComposition:
@pytest.fixture(scope="class")
def models(self):
return {
"my_model.sql": my_model_with_composition_sql,
"my_unit_test.yml": my_unit_test_yml,
}
@pytest.fixture(scope="class")
def macros(self):
return {"my_macros.sql": my_macro_sql}
def test_macro_composition_in_unit_test(self, project):
# Verify model works fine outside of unit testing
results = run_dbt(["run"])
assert len(results) == 1
# Test that a model with macro composition properly resolves macro names in unit tests
run_dbt(["test"])

View File

@@ -0,0 +1,87 @@
import pytest
from dbt.tests.util import run_dbt, run_dbt_and_capture
dbt_project_yml = """
vars:
columns_list_one:
- column_a
- column_b
columns_list_two:
- column_c
"""
my_model_one_variable_sql = """
{{ config(materialized='table') }}
-- {{ get_columns(include=var('columns_list_one'))}}
select 1 as id
"""
my_model_two_variables_sql = """
{{ config(materialized='table') }}
-- {{ get_columns(include=var('columns_list_one') + var('columns_list_two'))}}
select 1 as id
"""
my_macro_sql = """
{%- macro get_columns(include=[]) -%}
{%- for col in include -%}
{{ col }}{% if not loop.last %}, {% endif %}
{%- endfor -%}
{%- endmacro -%}
"""
my_unit_test_yml = """
unit_tests:
- name: my_unit_test
model: my_model
given: []
expect:
rows:
- {id: 1}
"""
class TestUnitTestOneVariables:
@pytest.fixture(scope="class")
def project_config_update(self):
return dbt_project_yml
@pytest.fixture(scope="class")
def models(self):
return {
"my_model.sql": my_model_one_variable_sql,
"my_unit_test.yml": my_unit_test_yml,
}
@pytest.fixture(scope="class")
def macros(self):
return {"my_macros.sql": my_macro_sql}
def test_one_variable_as_input_to_macro(self, project):
run_dbt_and_capture(["test"], expect_pass=True)
class TestUnitTestTwoVariables:
@pytest.fixture(scope="class")
def project_config_update(self):
return dbt_project_yml
@pytest.fixture(scope="class")
def models(self):
return {
"my_model.sql": my_model_two_variables_sql,
"my_unit_test.yml": my_unit_test_yml,
}
@pytest.fixture(scope="class")
def macros(self):
return {"my_macros.sql": my_macro_sql}
def test_two_variables_as_input_to_macro(self, project):
# Verify model works fine outside of unit testing
results = run_dbt(["run"])
assert len(results) == 1
run_dbt(["test"])