Compare commits

...

80 Commits

Author SHA1 Message Date
Kshitij Aranke
0dcad7d2dd Remove homebrew services (#11456)
(cherry picked from commit 9e6facc4d1035aed0055061040d3b39299a5fefb)
2025-04-03 13:15:56 +00:00
Kshitij Aranke
b02fd3a781 Add retry to macos postgres setup (#11432) (#11433)
* Retry macos postgres setup

* Retry command instead of step

* Use setup_db.sh

* Simplify macos postgres setup
2025-03-27 18:43:55 +00:00
github-actions[bot]
f407ae629f fix postgres github action using vibe coding (#11409) (#11410)
(cherry picked from commit 6a954e2d24f07494eca869367cff4d7abadecc07)

Co-authored-by: Kshitij Aranke <kshitij.aranke@dbtlabs.com>
2025-03-20 14:27:16 -05:00
Emily Rockman
a5b38501e2 [backport 1.4] handle ubuntu 20.04 deprecation (#11354) (#11360)
* handle ubuntu 20.04 deprecation (#11354)

* update ubuntu 20.04 to 24.04

* updates to ubuntu-latest instead

* try postgres update

* Change owner of db creation script so postgres can run it.

* Add sudos.

* Add debug logging.

* Set execute bit on scripts.

* More debug logging.

* try a service

* clean up and split the integrations tests by os

---------

Co-authored-by: Peter Allen Webb <peter.webb@dbtlabs.com>
# Conflicts:
#	.github/actions/setup-postgres-linux/action.yml
#	.github/workflows/main.yml
#	.github/workflows/structured-logging-schema-check.yml

* conflict fixes
2025-03-04 14:04:04 -06:00
Emily Rockman
c3f97d8f55 upgrade macos version (#10974) (#10980)
* upgrade to macos-latest

* force link
# Conflicts:
#	.github/actions/setup-postgres-macos/action.yml
#	.github/workflows/main.yml
2024-11-06 12:54:08 -06:00
Peter Webb
02634f3e4a Remove Python 3.8 from various places (#10861) (#10873)
* Remove Python 3.8 from various places

* Add changelog entry.

---------

Co-authored-by: Gerda Shank <gerda@dbtlabs.com>
2024-10-18 17:13:53 -04:00
Kshitij Aranke
623e595259 [Backport 1.4.latest] [Tidy First] Remove wheel and twine from dev-requirements.txt (#10204) 2024-05-21 23:38:16 +01:00
Kshitij Aranke
ac9a93c2c6 [Backport 1.4.latest] Fix #9907: Add retry to tox to reduce flaky tests due to network failures (#10183) 2024-05-21 21:02:44 +01:00
Emily Rockman
4d5438244f pin macos test runners to macos-12 (#10031) (#10038)
# Conflicts:
#	.github/workflows/main.yml

Co-authored-by: Mike Alfare <13974384+mikealfare@users.noreply.github.com>
2024-04-25 10:05:55 -05:00
Emily Rockman
3ede704b72 [1.7] Fix Workflow Deprecations (#9799) (#9910)
* update to current node version

* remember codecov

* ensure uniquness in name

---------

Co-authored-by: Kshitij Aranke <kshitij.aranke@dbtlabs.com>
# Conflicts:
#	.github/actions/latest-wrangler/examples/example_workflow.yml
#	.github/workflows/main.yml
#	.github/workflows/model_performance.yml
#	.github/workflows/schema-check.yml
2024-04-17 07:46:59 -05:00
Emily Rockman
8b7e6c7bf0 be less explicit (#9936) (#9943) 2024-04-15 12:26:58 -05:00
Emily Rockman
a7cb4c76d2 update to wrk for all versions (#9916) (#9922)
# Conflicts:
#	.github/workflows/main.yml
2024-04-15 07:59:56 -05:00
Kshitij Aranke
f1a47afee0 [Backport 1.4.latest] Pin pytest in dev-requirements.txt (#9477) 2024-01-30 16:45:44 +00:00
github-actions[bot]
5367774aa4 test pinning ddtrace (#9090) (#9096)
(cherry picked from commit 3902137dfc)

Co-authored-by: Michelle Ark <MichelleArk@users.noreply.github.com>
2023-11-15 17:04:54 -06:00
FishtownBuildBot
d9c369ac13 [Automated] Merged prep-release/1.4.9_6486428443 into target 1.4.latest during release process 2023-10-11 15:02:37 -04:00
Github Build Bot
a28004584c Bumping version to 1.4.9 and generate changelog 2023-10-11 18:22:30 +00:00
Michelle Ark
e1cf58671d [Backport 1.4.latest] respect project root when loading seeds (#8762) (#8806) 2023-10-11 18:46:02 +01:00
FishtownBuildBot
98a84b93c2 [Automated] Merged prep-release/1.4.8_6252959299 into target 1.4.latest during release process 2023-09-20 14:35:42 -05:00
Github Build Bot
8d74b98a0f Bumping version to 1.4.8 and generate changelog 2023-09-20 19:05:36 +00:00
Emily Rockman
8f4054bef6 [BACKPORT] 1.4.latest - 7630 (#8604)
* Exclude some profile fields from Jinja rendering when they are not valid Jinja. (#7630)

* CT-2583: Exclude some profile fields from Jinja rendering.

* CT-2583: Add functional test.

* CT-2583: Change approach to password jinja detection

* CT-2583: Extract string constant and add additional checks

* CT-2583: Improve unit test coverage

* Remove unneeded unit test.

---------

Co-authored-by: Peter Webb <peter.webb@dbtlabs.com>
2023-09-19 14:46:41 -04:00
Gerda Shank
845aeb697d Fix snapshot success message to display "INSERT 0 1" (for example) instead of success (#8524) (#8532) 2023-08-31 14:55:56 -04:00
github-actions[bot]
9104c2a77c add env vars to tox.ini (#8365) (#8367)
* add env vars to tox.ini

* revert test

(cherry picked from commit b7aee3f5a4)

Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com>
2023-08-11 15:41:37 -05:00
FishtownBuildBot
8731f9d2e8 [Automated] Merged prep-release/1.4.7_5764146089 into target 1.4.latest during release process 2023-08-04 11:33:45 -05:00
Github Build Bot
1e4d93ce31 Bumping version to 1.4.7 and generate changelog 2023-08-04 15:57:51 +00:00
github-actions[bot]
0895b872d3 [Backport 1.4.latest] Hotfix for 372: Use JSONEncoder in json.dumps (#8161)
Co-authored-by: Kshitij Aranke <kshitij.aranke@dbtlabs.com>
2023-07-21 16:06:20 -05:00
Emily Rockman
eac23890c5 add env vars for datadog ci visibility (#8097) (#8105)
* add env vars for datadog ci visibility

* modify pytest command for tracing

* fix posargs

* move env vars to job that needs them

* add test repeater to DD

* swap flags
# Conflicts:
#	.github/workflows/test-repeater.yml
2023-07-18 08:17:24 -05:00
Emily Rockman
bf756d27d4 pin click (#8050) (#8053)
* pin click

* changelog
# Conflicts:
#	core/setup.py
2023-07-07 12:38:15 -05:00
Emily Rockman
1273fc69d7 [Backport] 1.4.latest gh deprecations (#7636)
* cherry pick 7fbeced315

* cherry pick 43e24c5ae6
2023-05-16 11:48:53 -05:00
FishtownBuildBot
59c937748d [Automated] Merged prep-release/1.4.6_4745746349 into target 1.4.latest during release process 2023-04-19 12:09:35 -05:00
Github Build Bot
ccb3742fc5 Bumping version to 1.4.6 and generate changelog 2023-04-19 16:40:39 +00:00
Quigley Malcolm
7f68da16b7 [BACKPORT 1.4.latest] Improved event log msg stringification (#7342)
* Add tests for logging jinja2.Undefined objects

[CT-2259](https://github.com/dbt-labs/dbt-core/issues/7108) identifies
an issue wherein dbt-core 1.0-1.3 raise errors if a jinja2.Undefined
object is attempted to be logged. This generally happened in the form
of `{{ log(undefined_variable, info=True) }}`. This commit adding this
test exists for two reasons
1. Ensure we don't have a regression in this going forward
2. Exist as a commit to be used for backport fixes for dbt-core 1.0-1.3

* Add tests for checking `DBT_ENV_SECRET_`s don't break logging

[CT-1783](https://github.com/dbt-labs/dbt-core/issues/6568) describes
a bug in dbt-core 1.0-1.3 wherein when a `DBT_ENV_SECRET_` all
`{{ log("logging stuff", info=True) }}` invocations break. This commit
adds a test for this for two reasons:
1. Ensure we don't regress to this behavior going forward
2. Act as a base commit for making the backport fixes to dbt-core 1.0-1.3

* Log warning when event serialization fails in `msg_to_dict`

This commit updates the `msg_to_dict` exception handling path to
fire a warning level event instead of raising an exception.

* Add changie info for changes

* Add test to check exception handling of `msg_to_dict`

* Ensure all events with self.msg stringify self.msg

---------

Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com>
2023-04-18 13:19:59 -07:00
github-actions[bot]
5f696eb989 sqlparse <0.4.4 (#7394) (#7397)
(cherry picked from commit 57e9096816)

Co-authored-by: Michelle Ark <MichelleArk@users.noreply.github.com>
2023-04-18 12:48:03 -04:00
github-actions[bot]
7d1c7518f8 Raise upper pin for hologram to 0.0.16 (#7221) (#7290)
(cherry picked from commit b718c537a7)

Co-authored-by: Gerda Shank <gerda@dbtlabs.com>
2023-04-07 15:34:47 -04:00
Jeremy Cohen
30f8c6269a Make use of hashlib.md5() FIPS compliant (#6982) (#7074)
Signed-off-by: Niels Pardon <par@zurich.ibm.com>
Co-authored-by: Niels Pardon <mail@niels-pardon.de>
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com>
2023-04-04 15:53:29 +02:00
FishtownBuildBot
65fcbe7b79 [Automated] Merged prep-release/1.4.5_4389129737 into target 1.4.latest during release process 2023-03-10 17:30:19 -06:00
Github Build Bot
f745e7c823 Bumping version to 1.4.5 and generate changelog 2023-03-10 23:03:03 +00:00
colin-rogers-dbt
14f966c24f backport 7115 (#7152) 2023-03-10 10:46:39 -08:00
Doug Beatty
d0a6ea96e4 Add new index.html and changelog yaml files from dbt-docs (#7142)
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
2023-03-08 13:26:27 -07:00
github-actions[bot]
f3f1244eb0 [CT-1959]: moving simple_seed tests to adapter zone (#6859) (#7131)
* Formatting

* Changelog entry

* Rename to BaseSimpleSeedColumnOverride

* Better error handling

* Update test to include the BOM test

* Cleanup and formating

* Unused import remove

* nit line

* Pr comments

(cherry picked from commit 4c63b630de)

Co-authored-by: Neelesh Salian <nssalian@users.noreply.github.com>
2023-03-07 08:37:59 -08:00
github-actions[bot]
0c067350da CT 2057 Fix compilation logic for ephemeral nodes (#7023) (#7037)
* Don't overwrite sql in extra_ctes when compiling (rendering) nodes

(cherry picked from commit 70c26f5c74)

Co-authored-by: Gerda Shank <gerda@dbtlabs.com>
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com>
2023-03-01 07:53:49 -05:00
FishtownBuildBot
57a6963913 [Automated] Merged prep-release/1.4.4_4294080763 into target 1.4.latest during release process 2023-02-28 09:07:01 -06:00
Github Build Bot
d2c9ee46a1 Bumping version to 1.4.4 and generate changelog 2023-02-28 14:38:35 +00:00
Sam Debruyn
89fa60d86a fix: add pytz dependency (#7078) 2023-02-28 09:07:34 -05:00
FishtownBuildBot
e39505fec6 [Automated] Merged prep-release/1.4.3_4258471710 into target 1.4.latest during release process 2023-02-23 20:11:31 -06:00
Github Build Bot
c82795a99f Bumping version to 1.4.3 and generate changelog 2023-02-24 01:35:10 +00:00
Neelesh Salian
f0f4af5c8d Fixing the release secret (#7043) 2023-02-23 17:31:04 -08:00
github-actions[bot]
3fe1502c15 Fix regression in semver comparison logic (#7040) (#7041)
(cherry picked from commit 915585c36e)

Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com>
2023-02-23 16:33:28 -08:00
FishtownBuildBot
cc2f259108 [Automated] Merged prep-release/1.4.2_4255074059 into target 1.4.latest during release process 2023-02-23 11:41:42 -06:00
Github Build Bot
51d216186d Bumping version to 1.4.2 and generate changelog 2023-02-23 17:14:29 +00:00
leahwicz
bdf7c1ff82 Revert "CT 2057 Fix compilation logic for ephemeral nodes (#7023) (#7032)" (#7035)
This reverts commit f23dbb6f6b.
2023-02-23 12:09:33 -05:00
github-actions[bot]
f23dbb6f6b CT 2057 Fix compilation logic for ephemeral nodes (#7023) (#7032)
* Don't overwrite sql in extra_ctes when compiling (rendering) nodes

(cherry picked from commit 70c26f5c74)

Co-authored-by: Gerda Shank <gerda@dbtlabs.com>
2023-02-22 18:05:00 -05:00
FishtownBuildBot
ec17ecec86 [Automated] Merged prep-release/1.4.2rc2_4246496113 into target 1.4.latest during release process 2023-02-22 14:36:47 -06:00
Github Build Bot
a3d326404d Bumping version to 1.4.2rc2 and generate changelog 2023-02-22 20:06:40 +00:00
Peter Webb
c785f320ea CT-2160: Fix logbook (legacy logging) regression (#7029)
* CT-2160: Fix logbook (legacy logging) regression

* CT-2160: Changelog entry
2023-02-22 14:19:10 -05:00
FishtownBuildBot
1fe0c32890 [Automated] Merged prep-release/1.4.2rc1_4187540534 into target 1.4.latest during release process 2023-02-15 14:15:08 -06:00
Github Build Bot
11752edef8 Bumping version to 1.4.2rc1 and generate changelog 2023-02-15 19:47:50 +00:00
github-actions[bot]
16cb498b56 [Backport 1.4.latest] Set relation_name in tests at compile time (#6978)
* Set relation_name in tests at compile time (#6949)

(cherry picked from commit 480e0e55c5)

* Tweak artifact test

---------

Co-authored-by: Gerda Shank <gerda@dbtlabs.com>
2023-02-14 18:33:12 -05:00
Gerda Shank
03f6655f1a Remove empty changie file (#6977) 2023-02-14 15:52:35 -05:00
Gerda Shank
8dc3d0a531 Fix state comparison with disabled exposure (or metric) (#6967) 2023-02-13 21:55:18 -05:00
github-actions[bot]
c172687736 CT 2000 fix semver prerelease comparisons (#6838) (#6958)
* Modify semver.py to not use packaging.version.parse

* Changie

(cherry picked from commit d9424cc710)

Co-authored-by: Gerda Shank <gerda@dbtlabs.com>
2023-02-13 22:39:59 +01:00
Emily Rockman
8a31eab181 Add most recent dbt-docs changes (#6923) (#6925)
* Add new index.html and changelog yaml files from dbt-docs

* Update .changes/unreleased/Docs-20230209-082901.yaml

---------

Co-authored-by: FishtownBuildBot <77737458+FishtownBuildBot@users.noreply.github.com>
2023-02-10 11:14:33 -06:00
Peter Webb
4d0ee2fc47 Ensure flush() after logging write() (#6909) (#6922)
* ct-2063: Ensure flush after logging, by using Python's logging subsystem directly

* ct-2063: Add changelog entry
2023-02-09 10:48:30 -05:00
github-actions[bot]
ddb2f0f71d [Backport 1.4.latest] Add back depends_on for seeds - only macros, never nodes (#6920)
* Add back `depends_on` for seeds - only `macros`, never `nodes` (#6851)

* Extend functional tests for seeds w hooks

* Add MacroDependsOn to seeds, raise exception for other deps

* Add changelog entry

* Fix unit tests

* Update upgrade_seed_content

* Cleanup

* Regen manifest v8 schema. Fix tests

* Be less magical

* PR feedback

(cherry picked from commit 298bf8a1d4)

* Update manifest v8 for v1.4.x

* Cleanup

---------

Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com>
2023-02-09 12:37:46 +01:00
Emily Rockman
2bb2e73df9 1.4 regression: Check if status has node attribute (#6899) (#6904)
* check for node

* add changelog

* add test for regression
2023-02-08 15:00:32 -06:00
Peter Webb
790fecab92 CT-1917: Fix a regression in the behavior of the -q/--quiet cli parameter (#6886) (#6889) 2023-02-08 10:38:42 -05:00
Emily Rockman
f6926c0ed6 [BACKPORT] 1.4.latest update regex to match all iterations (#6839) (#6846)
* update regex to match all iterations (#6839)

* update regex to match all iterations

* convert to num to match all adapters

* add comments, remove extra .

* clarify with more comments

* Update .bumpversion.cfg

Co-authored-by: Nathaniel May <nathaniel.may@fishtownanalytics.com>

---------

Co-authored-by: Nathaniel May <nathaniel.may@fishtownanalytics.com>
# Conflicts:
#	.bumpversion.cfg

* put back correct version

* put back correct version
2023-02-06 13:54:34 -06:00
Alexander Smolyakov
52be5ffaa6 [CI/CD] Backport release workflow to 1.4.latest (#6793)
* [CI/CD] Update release workflow and introduce workflow for nightly releases (#6602)

* Add release workflows

* Update nightly-release.yml

* Set default `test_run` value to `true`

* Update .bumpversion.cfg

* Resolve review comment

- Update workflow docs
- Change workflow name
- Set `test_run` default value to `true`

* Update Slack secret

* PyPI

* Update release workflow (#6778)

- Update AWS secrets
- Rework condition for Slack notification

---------

Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com>
2023-02-01 14:20:33 -06:00
Emily Rockman
31bfff81d9 fix contributor list generation (#6799) (#6808) 2023-01-31 21:39:13 -06:00
github-actions[bot]
3fbfa6e6f3 [Backport 1.4.latest] CT 1894 log partial parsing var changes and sort cli vars before hashing (#6758)
* CT 1894 log partial parsing var changes and sort cli vars before hashing (#6713)

* Log information about vars_hash, normalize cli_vars before hashing

* Changie
2023-01-26 15:04:25 -05:00
github-actions[bot]
165035692f Bumping version to 1.4.1 and generate changelog (#6743)
* Bumping version to 1.4.1rc1 and generate CHANGELOG

* Bumping version to 1.4.1 and generate CHANGELOG (#6744)

Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>

* Adjust docker version image

Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com>
2023-01-25 20:48:44 -05:00
github-actions[bot]
80f76e9e6e change exposure_content to source_content (#6739) (#6740)
* change `exposure_content` to `source_content`

* Adding changelog

Co-authored-by: Leah Antkiewicz <leah.antkiewicz@fishtownanalytics.com>
(cherry picked from commit b0651b13b5)

Co-authored-by: Matthew Beall <matthew@beall.org>
2023-01-25 20:11:38 -05:00
github-actions[bot]
a2e7249c44 Bumping version to 1.4.0 and generate CHANGELOG (#6727)
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
2023-01-25 10:17:36 -05:00
github-actions[bot]
9700ff1866 CT 1886 include adapter_response in NodeFinished log message (#6709) (#6714)
* Include adapter_response in run_result in NodeFinished log event

* Changie

(cherry picked from commit e2ccf011d9)

Co-authored-by: Gerda Shank <gerda@dbtlabs.com>
2023-01-24 15:38:51 -05:00
colin-rogers-dbt
e61a39f27f mv on_schema_change tests -> "adapter zone" (#6618) (#6686)
* mv `on_schema_change` tests -> "adapter zone" (#6618)

* Mv incremental on_schema_change tests to 'adapter zone'

* Use type_string()

* Cleanup

* mv `on_schema_change` tests -> "adapter zone" (#6618)

* Mv incremental on_schema_change tests to 'adapter zone'

* Use type_string()

* Cleanup

Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com>
2023-01-20 12:26:32 -08:00
github-actions[bot]
d934e713db Bumping version to 1.4.0rc2 and generate CHANGELOG (#6661)
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
2023-01-19 12:13:06 -05:00
github-actions[bot]
ef9bb925d3 add backwards compatibility and default argument for incremental_predicates (#6628) (#6660)
* add backwards compatibility and default argument

* changie <3

* Update .changes/unreleased/Fixes-20230117-101342.yaml

Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com>
(cherry picked from commit f841a7ca76)

Co-authored-by: dave-connors-3 <73915542+dave-connors-3@users.noreply.github.com>
2023-01-19 10:24:49 -05:00
github-actions[bot]
f73359b87c [Backport 1.4.latest] convert 062_defer_state_tests (#6657)
* convert 062_defer_state_tests (#6616)

* Fix --favor-state flag

* Convert 062_defer_state_tests

* Revert "Fix --favor-state flag"

This reverts commit ccbdcbad98b26822629364e6fdbd2780db0c20d3.

* Reformat

* Revert "Revert "Fix --favor-state flag""

This reverts commit fa9d2a09d693b1870bd724a694fce2883748c987.

(cherry picked from commit 07a004b301)

* Add changelog entry

Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com>
2023-01-19 13:27:23 +01:00
Emily Rockman
b4706c4dec finish message rename in types.proto (#6594) (#6596)
* finish message rename in types.proto

* add new parameter
2023-01-13 10:20:34 -06:00
github-actions[bot]
b46d35c13f Call update_event_status earlier + rename an event (#6572) (#6591)
* Rename HookFinished -> FinishedRunningStats

* Move update_event_status earlier when node finishes

* Add changelog entry

* Add update_event_status for skip

* Update changelog entry

(cherry picked from commit 86e8722cd8)

Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com>
2023-01-13 11:53:14 +01:00
github-actions[bot]
eba90863ed Bumping version to 1.4.0rc1 and generate CHANGELOG (#6569)
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
2023-01-10 22:04:55 -05:00
209 changed files with 3948 additions and 3133 deletions

View File

@@ -1,13 +1,15 @@
[bumpversion]
current_version = 1.4.0b1
parse = (?P<major>\d+)
\.(?P<minor>\d+)
\.(?P<patch>\d+)
((?P<prekind>a|b|rc)
(?P<pre>\d+) # pre-release version num
current_version = 1.4.9
parse = (?P<major>[\d]+) # major version number
\.(?P<minor>[\d]+) # minor version number
\.(?P<patch>[\d]+) # patch version number
(((?P<prekind>a|b|rc) # optional pre-release type
?(?P<num>[\d]+?)) # optional pre-release version number
\.?(?P<nightly>[a-z0-9]+\+[a-z]+)? # optional nightly release indicator
)?
serialize =
{major}.{minor}.{patch}{prekind}{pre}
{major}.{minor}.{patch}{prekind}{num}.{nightly}
{major}.{minor}.{patch}{prekind}{num}
{major}.{minor}.{patch}
commit = False
tag = False
@@ -21,9 +23,11 @@ values =
rc
final
[bumpversion:part:pre]
[bumpversion:part:num]
first_value = 1
[bumpversion:part:nightly]
[bumpversion:file:core/setup.py]
[bumpversion:file:core/dbt/version.py]

View File

@@ -1,9 +1,14 @@
## dbt-core 1.4.0-b1 - December 15, 2022
## dbt-core 1.4.0 - January 25, 2023
### Breaking Changes
- Cleaned up exceptions to directly raise in code. Also updated the existing exception to meet PEP guidelines.Removed use of all exception functions in the code base and marked them all as deprecated to be removed next minor release. ([#6339](https://github.com/dbt-labs/dbt-core/issues/6339), [#6393](https://github.com/dbt-labs/dbt-core/issues/6393), [#6460](https://github.com/dbt-labs/dbt-core/issues/6460))
### Features
- Added favor-state flag to optionally favor state nodes even if unselected node exists ([#2968](https://github.com/dbt-labs/dbt-core/issues/2968))
- Added favor-state flag to optionally favor state nodes even if unselected node exists ([#5016](https://github.com/dbt-labs/dbt-core/issues/5016))
- Update structured logging. Convert to using protobuf messages. Ensure events are enriched with node_info. ([#5610](https://github.com/dbt-labs/dbt-core/issues/5610))
- incremental predicates ([#5680](https://github.com/dbt-labs/dbt-core/issues/5680))
- Friendlier error messages when packages.yml is malformed ([#5486](https://github.com/dbt-labs/dbt-core/issues/5486))
- Allow partitions in external tables to be supplied as a list ([#5929](https://github.com/dbt-labs/dbt-core/issues/5929))
- extend -f flag shorthand for seed command ([#5990](https://github.com/dbt-labs/dbt-core/issues/5990))
@@ -12,7 +17,10 @@
- Added an md5 function to the base context ([#6246](https://github.com/dbt-labs/dbt-core/issues/6246))
- Exposures support metrics in lineage ([#6057](https://github.com/dbt-labs/dbt-core/issues/6057))
- Add support for Python 3.11 ([#6147](https://github.com/dbt-labs/dbt-core/issues/6147))
- incremental predicates ([#5680](https://github.com/dbt-labs/dbt-core/issues/5680))
- Making timestamp optional for metrics ([#6398](https://github.com/dbt-labs/dbt-core/issues/6398))
- The meta configuration field is now included in the node_info property of structured logs. ([#6216](https://github.com/dbt-labs/dbt-core/issues/6216))
- Adds buildable selection mode ([#6365](https://github.com/dbt-labs/dbt-core/issues/6365))
- --warn-error-options: Treat warnings as errors for specific events, based on user configuration ([#6165](https://github.com/dbt-labs/dbt-core/issues/6165))
### Fixes
@@ -21,11 +29,25 @@
- check length of args of python model function before accessing it ([#6041](https://github.com/dbt-labs/dbt-core/issues/6041))
- Add functors to ensure event types with str-type attributes are initialized to spec, even when provided non-str type params. ([#5436](https://github.com/dbt-labs/dbt-core/issues/5436))
- Allow hooks to fail without halting execution flow ([#5625](https://github.com/dbt-labs/dbt-core/issues/5625))
- fix missing f-strings, convert old .format() messages to f-strings for consistency ([#6241](https://github.com/dbt-labs/dbt-core/issues/6241))
- Clarify Error Message for how many models are allowed in a Python file ([#6245](https://github.com/dbt-labs/dbt-core/issues/6245))
- Fix typo in util.py ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904))
- After this, will be possible to use default values for dbt.config.get ([#6309](https://github.com/dbt-labs/dbt-core/issues/6309))
- Use full path for writing manifest ([#6055](https://github.com/dbt-labs/dbt-core/issues/6055))
- [CT-1284] Change Python model default materialization to table ([#6345](https://github.com/dbt-labs/dbt-core/issues/6345))
- add pre-commit install to make dev script in Makefile ([#6269](https://github.com/dbt-labs/dbt-core/issues/6269))
- Late-rendering for `pre_` and `post_hook`s in `dbt_project.yml` ([#6411](https://github.com/dbt-labs/dbt-core/issues/6411))
- [CT-1284] Change Python model default materialization to table ([#5989](https://github.com/dbt-labs/dbt-core/issues/5989))
- [CT-1591] Don't parse empty Python files ([#6345](https://github.com/dbt-labs/dbt-core/issues/6345))
- Repair a regression which prevented basic logging before the logging subsystem is completely configured. ([#6434](https://github.com/dbt-labs/dbt-core/issues/6434))
- fix docs generate --defer by adding defer_to_manifest to before_run ([#6488](https://github.com/dbt-labs/dbt-core/issues/6488))
- Bug when partial parsing with an empty schema file ([#4850](https://github.com/dbt-labs/dbt-core/issues/4850))
- Fix DBT_FAVOR_STATE env var ([#5859](https://github.com/dbt-labs/dbt-core/issues/5859))
- Restore historical behavior of certain disabled test messages, so that they are at the less obtrusive debug level, rather than the warning level. ([#6501](https://github.com/dbt-labs/dbt-core/issues/6501))
- Bump mashumuro version to get regression fix and add unit test to verify that fix. ([#6428](https://github.com/dbt-labs/dbt-core/issues/6428))
- Call update_event_status earlier for node results. Rename event 'HookFinished' -> FinishedRunningStats ([#6571](https://github.com/dbt-labs/dbt-core/issues/6571))
- Provide backward compatibility for `get_merge_sql` arguments ([#6625](https://github.com/dbt-labs/dbt-core/issues/6625))
- Fix behavior of --favor-state with --defer ([#6617](https://github.com/dbt-labs/dbt-core/issues/6617))
- Include adapter_response in NodeFinished run_result log event ([#6703](https://github.com/dbt-labs/dbt-core/issues/6703))
### Docs
@@ -34,6 +56,7 @@
- ([dbt-docs/#5880](https://github.com/dbt-labs/dbt-docs/issues/5880))
- Fix rendering of sample code for metrics ([dbt-docs/#323](https://github.com/dbt-labs/dbt-docs/issues/323))
- Alphabetize `core/dbt/README.md` ([dbt-docs/#6368](https://github.com/dbt-labs/dbt-docs/issues/6368))
- Updated minor typos encountered when skipping profile setup ([dbt-docs/#6529](https://github.com/dbt-labs/dbt-docs/issues/6529))
### Under the Hood
@@ -57,6 +80,14 @@
- Fix intermittent database connection failure in Windows CI test ([#6394](https://github.com/dbt-labs/dbt-core/issues/6394))
- Refactor and clean up manifest nodes ([#6426](https://github.com/dbt-labs/dbt-core/issues/6426))
- Restore important legacy logging behaviors, following refactor which removed them ([#6437](https://github.com/dbt-labs/dbt-core/issues/6437))
- Treat dense text blobs as binary for `git grep` ([#6294](https://github.com/dbt-labs/dbt-core/issues/6294))
- Prune partial parsing logging events ([#6313](https://github.com/dbt-labs/dbt-core/issues/6313))
- Updating the deprecation warning in the metric attributes renamed event ([#6507](https://github.com/dbt-labs/dbt-core/issues/6507))
- [CT-1693] Port severity test to Pytest ([#6466](https://github.com/dbt-labs/dbt-core/issues/6466))
- [CT-1694] Deprecate event tracking tests ([#6467](https://github.com/dbt-labs/dbt-core/issues/6467))
- Reorganize structured logging events to have two top keys ([#6311](https://github.com/dbt-labs/dbt-core/issues/6311))
- Combine some logging events ([#1716](https://github.com/dbt-labs/dbt-core/issues/1716), [#1717](https://github.com/dbt-labs/dbt-core/issues/1717), [#1719](https://github.com/dbt-labs/dbt-core/issues/1719))
- Check length of escaped strings in the adapter test ([#6566](https://github.com/dbt-labs/dbt-core/issues/6566))
### Dependencies
@@ -64,25 +95,32 @@
- Bump black from 22.8.0 to 22.10.0 ([#6019](https://github.com/dbt-labs/dbt-core/pull/6019))
- Bump mashumaro[msgpack] from 3.0.4 to 3.1.1 in /core ([#6108](https://github.com/dbt-labs/dbt-core/pull/6108))
- Update colorama requirement from <0.4.6,>=0.3.9 to >=0.3.9,<0.4.7 in /core ([#6144](https://github.com/dbt-labs/dbt-core/pull/6144))
- Bump mashumaro[msgpack] from 3.1.1 to 3.2 in /core ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904))
- Bump mashumaro[msgpack] from 3.1.1 to 3.2 in /core ([#6375](https://github.com/dbt-labs/dbt-core/pull/6375))
- Update agate requirement from <1.6.4,>=1.6 to >=1.6,<1.7.1 in /core ([#6506](https://github.com/dbt-labs/dbt-core/pull/6506))
### Contributors
- [@NiallRees](https://github.com/NiallRees) ([#5859](https://github.com/dbt-labs/dbt-core/issues/5859))
- [@agpapa](https://github.com/agpapa) ([#6365](https://github.com/dbt-labs/dbt-core/issues/6365))
- [@andy-clapson](https://github.com/andy-clapson) ([dbt-docs/#5791](https://github.com/dbt-labs/dbt-docs/issues/5791))
- [@callum-mcdata](https://github.com/callum-mcdata) ([#6398](https://github.com/dbt-labs/dbt-core/issues/6398), [#6507](https://github.com/dbt-labs/dbt-core/issues/6507))
- [@chamini2](https://github.com/chamini2) ([#6041](https://github.com/dbt-labs/dbt-core/issues/6041))
- [@daniel-murray](https://github.com/daniel-murray) ([#2968](https://github.com/dbt-labs/dbt-core/issues/2968))
- [@dave-connors-3](https://github.com/dave-connors-3) ([#5990](https://github.com/dbt-labs/dbt-core/issues/5990))
- [@dbeatty10](https://github.com/dbeatty10) ([dbt-docs/#6368](https://github.com/dbt-labs/dbt-docs/issues/6368), [#6394](https://github.com/dbt-labs/dbt-core/issues/6394))
- [@daniel-murray](https://github.com/daniel-murray) ([#5016](https://github.com/dbt-labs/dbt-core/issues/5016))
- [@dave-connors-3](https://github.com/dave-connors-3) ([#5680](https://github.com/dbt-labs/dbt-core/issues/5680), [#5990](https://github.com/dbt-labs/dbt-core/issues/5990), [#6625](https://github.com/dbt-labs/dbt-core/issues/6625))
- [@dbeatty10](https://github.com/dbeatty10) ([#6411](https://github.com/dbt-labs/dbt-core/issues/6411), [dbt-docs/#6368](https://github.com/dbt-labs/dbt-docs/issues/6368), [#6394](https://github.com/dbt-labs/dbt-core/issues/6394), [#6294](https://github.com/dbt-labs/dbt-core/issues/6294), [#6566](https://github.com/dbt-labs/dbt-core/issues/6566))
- [@devmessias](https://github.com/devmessias) ([#6309](https://github.com/dbt-labs/dbt-core/issues/6309))
- [@eltociear](https://github.com/eltociear) ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904))
- [@eve-johns](https://github.com/eve-johns) ([#6068](https://github.com/dbt-labs/dbt-core/issues/6068))
- [@haritamar](https://github.com/haritamar) ([#6246](https://github.com/dbt-labs/dbt-core/issues/6246))
- [@jared-rimmer](https://github.com/jared-rimmer) ([#5486](https://github.com/dbt-labs/dbt-core/issues/5486))
- [@josephberni](https://github.com/josephberni) ([#2968](https://github.com/dbt-labs/dbt-core/issues/2968))
- [@josephberni](https://github.com/josephberni) ([#5016](https://github.com/dbt-labs/dbt-core/issues/5016))
- [@joshuataylor](https://github.com/joshuataylor) ([#6147](https://github.com/dbt-labs/dbt-core/issues/6147))
- [@justbldwn](https://github.com/justbldwn) ([#6245](https://github.com/dbt-labs/dbt-core/issues/6245))
- [@justbldwn](https://github.com/justbldwn) ([#6241](https://github.com/dbt-labs/dbt-core/issues/6241), [#6245](https://github.com/dbt-labs/dbt-core/issues/6245), [#6269](https://github.com/dbt-labs/dbt-core/issues/6269))
- [@luke-bassett](https://github.com/luke-bassett) ([#1350](https://github.com/dbt-labs/dbt-core/issues/1350))
- [@max-sixty](https://github.com/max-sixty) ([#5946](https://github.com/dbt-labs/dbt-core/issues/5946), [#5983](https://github.com/dbt-labs/dbt-core/issues/5983), [#6028](https://github.com/dbt-labs/dbt-core/issues/6028))
- [@mivanicova](https://github.com/mivanicova) ([#6488](https://github.com/dbt-labs/dbt-core/issues/6488))
- [@nshuman1](https://github.com/nshuman1) ([dbt-docs/#6529](https://github.com/dbt-labs/dbt-docs/issues/6529))
- [@paulbenschmidt](https://github.com/paulbenschmidt) ([dbt-docs/#5880](https://github.com/dbt-labs/dbt-docs/issues/5880))
- [@pgoslatara](https://github.com/pgoslatara) ([#5929](https://github.com/dbt-labs/dbt-core/issues/5929))
- [@racheldaniel](https://github.com/racheldaniel) ([#6201](https://github.com/dbt-labs/dbt-core/issues/6201))
- [@timle2](https://github.com/timle2) ([#4205](https://github.com/dbt-labs/dbt-core/issues/4205))
- [@dave-connors-3](https://github.com/dave-connors-3) ([#5680](https://github.com/dbt-labs/dbt-core/issues/5680))
- [@tmastny](https://github.com/tmastny) ([#6216](https://github.com/dbt-labs/dbt-core/issues/6216))

View File

@@ -1,6 +0,0 @@
kind: "Dependencies"
body: "Update pathspec requirement from ~=0.9.0 to >=0.9,<0.11 in /core"
time: 2022-09-23T00:06:46.00000Z
custom:
Author: dependabot[bot]
PR: "5917"

View File

@@ -1,6 +0,0 @@
kind: "Dependencies"
body: "Bump black from 22.8.0 to 22.10.0"
time: 2022-10-07T00:08:48.00000Z
custom:
Author: dependabot[bot]
PR: "6019"

View File

@@ -1,6 +0,0 @@
kind: "Dependencies"
body: "Bump mashumaro[msgpack] from 3.0.4 to 3.1.1 in /core"
time: 2022-10-20T00:07:53.00000Z
custom:
Author: dependabot[bot]
PR: "6108"

View File

@@ -1,6 +0,0 @@
kind: "Dependencies"
body: "Update colorama requirement from <0.4.6,>=0.3.9 to >=0.3.9,<0.4.7 in /core"
time: 2022-10-26T00:09:10.00000Z
custom:
Author: dependabot[bot]
PR: "6144"

View File

@@ -1,7 +0,0 @@
kind: "Dependencies"
body: "Bump mashumaro[msgpack] from 3.1.1 to 3.2 in /core"
time: 2022-12-05T00:21:18.00000Z
custom:
Author: dependabot[bot]
Issue: 4904
PR: 6375

View File

@@ -1,6 +0,0 @@
kind: Docs
body: minor doc correction
time: 2022-09-08T15:41:57.689162-04:00
custom:
Author: andy-clapson
Issue: "5791"

View File

@@ -1,6 +0,0 @@
kind: Docs
body: Generate API docs for new CLI interface
time: 2022-10-07T09:06:56.446078-05:00
custom:
Author: stu-k
Issue: "5528"

View File

@@ -1,5 +0,0 @@
kind: Docs
time: 2022-10-17T17:14:11.715348-05:00
custom:
Author: paulbenschmidt
Issue: "5880"

View File

@@ -1,6 +0,0 @@
kind: Docs
body: Fix rendering of sample code for metrics
time: 2022-11-16T15:57:43.204201+01:00
custom:
Author: jtcohen6
Issue: "323"

View File

@@ -1,6 +0,0 @@
kind: Docs
body: Alphabetize `core/dbt/README.md`
time: 2022-12-02T15:05:23.695333-07:00
custom:
Author: dbeatty10
Issue: "6368"

View File

@@ -1,7 +0,0 @@
kind: Features
body: Added favor-state flag to optionally favor state nodes even if unselected node
exists
time: 2022-04-08T16:54:59.696564+01:00
custom:
Author: daniel-murray josephberni
Issue: "5016"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Update structured logging. Convert to using protobuf messages. Ensure events are enriched with node_info.
time: 2022-08-17T15:48:57.225267-04:00
custom:
Author: gshank
Issue: "5610"

View File

@@ -1,7 +0,0 @@
kind: Features
body: incremental predicates
time: 2022-08-23T08:57:27.640804-05:00
custom:
Author: dave-connors-3
Issue: "5680"
PR: "5702"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Friendlier error messages when packages.yml is malformed
time: 2022-09-12T12:59:35.121188+01:00
custom:
Author: jared-rimmer
Issue: "5486"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Allow partitions in external tables to be supplied as a list
time: 2022-09-25T21:16:51.051239654+02:00
custom:
Author: pgoslatara
Issue: "5929"

View File

@@ -1,6 +0,0 @@
kind: Features
body: extend -f flag shorthand for seed command
time: 2022-10-03T11:07:05.381632-05:00
custom:
Author: dave-connors-3
Issue: "5990"

View File

@@ -1,7 +0,0 @@
kind: Features
body: This pulls the profile name from args when constructing a RuntimeConfig in lib.py,
enabling the dbt-server to override the value that's in the dbt_project.yml
time: 2022-11-02T15:00:03.000805-05:00
custom:
Author: racheldaniel
Issue: "6201"

View File

@@ -1,8 +0,0 @@
kind: Features
body: Adding tarball install method for packages. Allowing package tarball to be specified
via url in the packages.yaml.
time: 2022-11-07T10:50:18.464545-05:00
custom:
Author: timle2
Issue: "4205"
PR: "4689"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Added an md5 function to the base context
time: 2022-11-14T18:52:07.788593+02:00
custom:
Author: haritamar
Issue: "6246"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Exposures support metrics in lineage
time: 2022-11-30T11:29:13.256034-05:00
custom:
Author: michelleark
Issue: "6057"

View File

@@ -1,7 +0,0 @@
kind: Features
body: Add support for Python 3.11
time: 2022-12-06T15:07:04.753127+01:00
custom:
Author: joshuataylor MichelleArk jtcohen6
Issue: "6147"
PR: "6326"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Account for disabled flags on models in schema files more completely
time: 2022-09-16T10:48:54.162273-05:00
custom:
Author: emmyoop
Issue: "3992"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Add validation of enabled config for metrics, exposures and sources
time: 2022-10-10T11:32:18.752322-05:00
custom:
Author: emmyoop
Issue: "6030"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: check length of args of python model function before accessing it
time: 2022-10-11T16:07:15.464093-04:00
custom:
Author: chamini2
Issue: "6041"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Add functors to ensure event types with str-type attributes are initialized
to spec, even when provided non-str type params.
time: 2022-10-16T17:37:42.846683-07:00
custom:
Author: versusfacit
Issue: "5436"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Allow hooks to fail without halting execution flow
time: 2022-11-07T09:53:14.340257-06:00
custom:
Author: ChenyuLInx
Issue: "5625"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Clarify Error Message for how many models are allowed in a Python file
time: 2022-11-15T08:10:21.527884-05:00
custom:
Author: justbldwn
Issue: "6245"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: After this, will be possible to use default values for dbt.config.get
time: 2022-11-24T16:34:19.039512764-03:00
custom:
Author: devmessias
Issue: "6309"
PR: "6317"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Use full path for writing manifest
time: 2022-12-02T16:48:59.029519-05:00
custom:
Author: gshank
Issue: "6055"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: '[CT-1284] Change Python model default materialization to table'
time: 2022-12-13T11:26:20.550017-08:00
custom:
Author: aranke
Issue: "5989"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Repair a regression which prevented basic logging before the logging subsystem
is completely configured.
time: 2022-12-14T15:53:07.396512-05:00
custom:
Author: peterallenwebb
Issue: "6434"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Put black config in explicit config
time: 2022-09-27T19:42:59.241433-07:00
custom:
Author: max-sixty
Issue: "5946"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Added flat_graph attribute the Manifest class's deepcopy() coverage
time: 2022-09-29T13:44:06.275941-04:00
custom:
Author: peterallenwebb
Issue: "5809"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Add mypy configs so `mypy` passes from CLI
time: 2022-10-05T12:03:10.061263-07:00
custom:
Author: max-sixty
Issue: "5983"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Exception message cleanup.
time: 2022-10-07T09:46:27.682872-05:00
custom:
Author: emmyoop
Issue: "6023"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Add dmypy cache to gitignore
time: 2022-10-07T14:00:44.227644-07:00
custom:
Author: max-sixty
Issue: "6028"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Provide useful errors when the value of 'materialized' is invalid
time: 2022-10-13T18:19:12.167548-04:00
custom:
Author: peterallenwebb
Issue: "5229"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Fixed extra whitespace in strings introduced by black.
time: 2022-10-17T15:15:11.499246-05:00
custom:
Author: luke-bassett
Issue: "1350"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Clean up string formatting
time: 2022-10-17T15:58:44.676549-04:00
custom:
Author: eve-johns
Issue: "6068"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Remove the 'root_path' field from most nodes
time: 2022-10-28T10:48:37.687886-04:00
custom:
Author: gshank
Issue: "6171"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Combine certain logging events with different levels
time: 2022-10-28T11:03:44.887836-04:00
custom:
Author: gshank
Issue: "6173"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Convert threading tests to pytest
time: 2022-11-08T07:45:50.589147-06:00
custom:
Author: stu-k
Issue: "5942"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Convert postgres index tests to pytest
time: 2022-11-08T11:56:33.743042-06:00
custom:
Author: stu-k
Issue: "5770"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Convert use color tests to pytest
time: 2022-11-08T13:31:04.788547-06:00
custom:
Author: stu-k
Issue: "5771"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Add github actions workflow to generate high level CLI API docs
time: 2022-11-16T13:00:37.916202-06:00
custom:
Author: stu-k
Issue: "5942"

View File

@@ -1,8 +0,0 @@
kind: Under the Hood
body: Functionality-neutral refactor of event logging system to improve encapsulation
and modularity.
time: 2022-11-18T14:57:17.792622-05:00
custom:
Author: peterallenwebb
Issue: "6139"
PR: "6291"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Consolidate ParsedNode and CompiledNode classes
time: 2022-12-05T16:49:48.563583-05:00
custom:
Author: gshank
Issue: "6383"
PR: "6384"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Prevent doc gen workflow from running on forks
time: 2022-12-06T09:40:15.301984-06:00
custom:
Author: stu-k
Issue: "6386"
PR: "6390"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Fix intermittent database connection failure in Windows CI test
time: 2022-12-06T11:30:53.166009-07:00
custom:
Author: MichelleArk dbeatty10
Issue: "6394"
PR: "6395"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Refactor and clean up manifest nodes
time: 2022-12-11T21:42:40.560074-05:00
custom:
Author: gshank
Issue: "6426"
PR: "6427"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Restore important legacy logging behaviors, following refactor which removed
them
time: 2022-12-13T21:41:06.815133-05:00
custom:
Author: peterallenwebb
Issue: "6437"

8
.changes/1.4.1.md Normal file
View File

@@ -0,0 +1,8 @@
## dbt-core 1.4.1 - January 26, 2023
### Fixes
- [Regression] exposure_content referenced incorrectly ([#6738](https://github.com/dbt-labs/dbt-core/issues/6738))
### Contributors
- [@Mathyoub](https://github.com/Mathyoub) ([#6738](https://github.com/dbt-labs/dbt-core/issues/6738))

20
.changes/1.4.2.md Normal file
View File

@@ -0,0 +1,20 @@
## dbt-core 1.4.2 - February 23, 2023
### Fixes
- Sort cli vars before hashing for partial parsing ([#6710](https://github.com/dbt-labs/dbt-core/issues/6710))
- Remove pin on packaging and stop using it for prerelease comparisons ([#6834](https://github.com/dbt-labs/dbt-core/issues/6834))
- Readd depends_on.macros to SeedNode, to support seeds with hooks calling macros ([#6806](https://github.com/dbt-labs/dbt-core/issues/6806))
- Fix regression of --quiet cli parameter behavior ([#6749](https://github.com/dbt-labs/dbt-core/issues/6749))
- Ensure results from hooks contain nodes when processing them ([#6796](https://github.com/dbt-labs/dbt-core/issues/6796))
- Always flush stdout after logging ([#6901](https://github.com/dbt-labs/dbt-core/issues/6901))
- Set relation_name in test nodes at compile time ([#6930](https://github.com/dbt-labs/dbt-core/issues/6930))
- Fix disabled definition in WritableManifest ([#6752](https://github.com/dbt-labs/dbt-core/issues/6752))
- Fix regression in logbook log output ([#7028](https://github.com/dbt-labs/dbt-core/issues/7028))
### Docs
- Fix JSON path to overview docs ([dbt-docs/#366](https://github.com/dbt-labs/dbt-docs/issues/366))
### Contributors
- [@halvorlu](https://github.com/halvorlu) ([#366](https://github.com/dbt-labs/dbt-core/issues/366))

5
.changes/1.4.3.md Normal file
View File

@@ -0,0 +1,5 @@
## dbt-core 1.4.3 - February 24, 2023
### Fixes
- Fix semver comparison logic by ensuring numeric values ([#7039](https://github.com/dbt-labs/dbt-core/issues/7039))

8
.changes/1.4.4.md Normal file
View File

@@ -0,0 +1,8 @@
## dbt-core 1.4.4 - February 28, 2023
### Fixes
- add pytz dependency ([#7077](https://github.com/dbt-labs/dbt-core/issues/7077))
### Contributors
- [@sdebruyn](https://github.com/sdebruyn) ([#7077](https://github.com/dbt-labs/dbt-core/issues/7077))

19
.changes/1.4.5.md Normal file
View File

@@ -0,0 +1,19 @@
## dbt-core 1.4.5 - March 10, 2023
### Fixes
- Fix compilation logic for ephemeral nodes ([#6885](https://github.com/dbt-labs/dbt-core/issues/6885))
- allow adapters to change model name resolution in py models ([#7114](https://github.com/dbt-labs/dbt-core/issues/7114))
### Docs
- Fix JSON path to package overview docs ([dbt-docs/#390](https://github.com/dbt-labs/dbt-docs/issues/390))
### Under the Hood
- Moving simple_seed to adapter zone to help adapter test conversions ([#CT-1959](https://github.com/dbt-labs/dbt-core/issues/CT-1959))
### Contributors
- [@dbeatty10](https://github.com/dbeatty10) ([#390](https://github.com/dbt-labs/dbt-core/issues/390))
- [@nssalian](https://github.com/nssalian) ([#CT-1959](https://github.com/dbt-labs/dbt-core/issues/CT-1959))
- [@rlh1994](https://github.com/rlh1994) ([#390](https://github.com/dbt-labs/dbt-core/issues/390))

14
.changes/1.4.6.md Normal file
View File

@@ -0,0 +1,14 @@
## dbt-core 1.4.6 - April 19, 2023
### Fixes
- Make use of hashlib.md5() FIPS compliant ([#6900](https://github.com/dbt-labs/dbt-core/issues/6900))
- Improved failed event serialization handling and associated tests ([#7113](https://github.com/dbt-labs/dbt-core/issues/7113), [#7108](https://github.com/dbt-labs/dbt-core/issues/7108), [#6568](https://github.com/dbt-labs/dbt-core/issues/6568))
### Under the Hood
- Remove upper pin for hologram/jsonschema ([#6775](https://github.com/dbt-labs/dbt-core/issues/6775))
### Contributors
- [@QMalcolm](https://github.com/QMalcolm) ([#7113](https://github.com/dbt-labs/dbt-core/issues/7113), [#7108](https://github.com/dbt-labs/dbt-core/issues/7108), [#6568](https://github.com/dbt-labs/dbt-core/issues/6568))
- [@nielspardon](https://github.com/nielspardon) ([#6900](https://github.com/dbt-labs/dbt-core/issues/6900))

5
.changes/1.4.7.md Normal file
View File

@@ -0,0 +1,5 @@
## dbt-core 1.4.7 - August 04, 2023
### Dependencies
- Pin click>=7.0,<8.1.4 ([#8050](https://github.com/dbt-labs/dbt-core/pull/8050))

6
.changes/1.4.8.md Normal file
View File

@@ -0,0 +1,6 @@
## dbt-core 1.4.8 - September 20, 2023
### Fixes
- Exclude password fields from Jinja rendering. ([#7629](https://github.com/dbt-labs/dbt-core/issues/7629))
- Fix snapshot success message ([#7583](https://github.com/dbt-labs/dbt-core/issues/7583))

5
.changes/1.4.9.md Normal file
View File

@@ -0,0 +1,5 @@
## dbt-core 1.4.9 - October 11, 2023
### Fixes
- Enable seeds to be handled from stored manifest data ([#6875](https://github.com/dbt-labs/dbt-core/issues/6875))

View File

@@ -1,9 +0,0 @@
kind: Breaking Changes
body: Cleaned up exceptions to directly raise in code. Also updated the existing
exception to meet PEP guidelines.Removed use of all exception
functions in the code base and marked them all as deprecated to be removed next
minor release.
time: 2022-12-05T14:19:37.863032-06:00
custom:
Author: emmyoop
Issue: 6339 6393 6460

View File

@@ -1,6 +0,0 @@
kind: "Dependencies"
body: "Update agate requirement from <1.6.4,>=1.6 to >=1.6,<1.7.1 in /core"
time: 2023-01-04T00:03:06.00000Z
custom:
Author: dependabot[bot]
PR: 6506

View File

@@ -1,6 +0,0 @@
kind: Docs
body: Updated minor typos encountered when skipping profile setup
time: 2023-01-02T17:02:16.66596191-05:00
custom:
Author: nshuman1
Issue: 6529

View File

@@ -1,7 +0,0 @@
kind: Features
body: Making timestamp optional for metrics
time: 2022-12-07T09:17:22.571877-06:00
custom:
Author: callum-mcdata
Issue: "6398"
PR: "9400"

View File

@@ -1,7 +0,0 @@
kind: Features
body: The meta configuration field is now included in the node_info property of structured
logs.
time: 2022-12-30T10:48:20.486416-06:00
custom:
Author: tmastny
Issue: "6216"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Adds buildable selection mode
time: 2023-01-02T09:13:35.663627-05:00
custom:
Author: agpapa
Issue: "6365"

View File

@@ -1,7 +0,0 @@
kind: Features
body: '--warn-error-options: Treat warnings as errors for specific events, based on
user configuration'
time: 2023-01-04T18:10:03.203142-05:00
custom:
Author: MichelleArk
Issue: "6165"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: fix missing f-strings, convert old .format() messages to f-strings for consistency
time: 2022-11-13T10:41:50.009727-05:00
custom:
Author: justbldwn
Issue: "6241"
PR: "6243"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Fix typo in util.py
time: 2022-11-17T22:03:20.4836855+09:00
custom:
Author: eltociear
Issue: "4904"
PR: "6037"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: add pre-commit install to make dev script in Makefile
time: 2022-12-12T11:59:12.175136-05:00
custom:
Author: justbldwn
Issue: "6269"
PR: "6417"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Late-rendering for `pre_` and `post_hook`s in `dbt_project.yml`
time: 2022-12-13T09:26:55.11397-07:00
custom:
Author: dbeatty10
Issue: "6411"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: '[CT-1591] Don''t parse empty Python files'
time: 2022-12-13T11:39:15.818464-08:00
custom:
Author: aranke
Issue: "6345"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: fix docs generate --defer by adding defer_to_manifest to before_run
time: 2022-12-26T01:02:11.630614+01:00
custom:
Author: mivanicova
Issue: "6488"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Bug when partial parsing with an empty schema file
time: 2023-01-01T22:34:05.97322-05:00
custom:
Author: gshank
Issue: "4850"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Fix DBT_FAVOR_STATE env var
time: 2023-01-04T14:10:47.637495-08:00
custom:
Author: NiallRees
Issue: "5859"
PR: "6392"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Restore historical behavior of certain disabled test messages, so that they
are at the less obtrusive debug level, rather than the warning level.
time: 2023-01-09T16:12:54.064875-05:00
custom:
Author: peterallenwebb
Issue: "6501"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Bump mashumuro version to get regression fix and add unit test to verify that
fix.
time: 2023-01-10T12:41:32.339631-05:00
custom:
Author: peterallenwebb
Issue: "6428"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Treat dense text blobs as binary for `git grep`
time: 2022-12-19T19:34:35.890275-07:00
custom:
Author: dbeatty10
Issue: "6294"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Prune partial parsing logging events
time: 2022-12-21T12:19:04.7402-05:00
custom:
Author: gshank
Issue: "6313"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Updating the deprecation warning in the metric attributes renamed event
time: 2023-01-04T15:52:57.916398-06:00
custom:
Author: callum-mcdata
Issue: "6507"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: '[CT-1693] Port severity test to Pytest'
time: 2023-01-06T11:28:55.800547-08:00
custom:
Author: aranke
Issue: "6466"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: '[CT-1694] Deprecate event tracking tests'
time: 2023-01-06T11:44:12.210821-08:00
custom:
Author: aranke
Issue: "6467"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Reorganize structured logging events to have two top keys
time: 2023-01-09T09:59:07.842187-05:00
custom:
Author: gshank
Issue: "6311"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Combine some logging events
time: 2023-01-10T11:42:33.580756-05:00
custom:
Author: gshank
Issue: 1716 1717 1719

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Check length of escaped strings in the adapter test
time: 2023-01-10T14:56:48.044198-07:00
custom:
Author: dbeatty10
Issue: "6566"

View File

@@ -0,0 +1,6 @@
kind: Under the Hood
body: Remove support and testing for Python 3.8, which is now EOL.
time: 2024-10-16T14:40:56.451972-04:00
custom:
Author: gshank peterallenwebb
Issue: "10861"

View File

@@ -97,22 +97,28 @@ footerFormat: |
{{- /* we only want to include non-core team contributors */}}
{{- if not (has $authorLower $core_team)}}
{{- $changeList := splitList " " $change.Custom.Author }}
{{- /* Docs kind link back to dbt-docs instead of dbt-core issues */}}
{{- $IssueList := list }}
{{- $changeLink := $change.Kind }}
{{- if or (eq $change.Kind "Dependencies") (eq $change.Kind "Security") }}
{{- $changeLink = "[#nbr](https://github.com/dbt-labs/dbt-core/pull/nbr)" | replace "nbr" $change.Custom.PR }}
{{- else if eq $change.Kind "Docs"}}
{{- $changeLink = "[dbt-docs/#nbr](https://github.com/dbt-labs/dbt-docs/issues/nbr)" | replace "nbr" $change.Custom.Issue }}
{{- $changes := splitList " " $change.Custom.PR }}
{{- range $issueNbr := $changes }}
{{- $changeLink := "[#nbr](https://github.com/dbt-labs/dbt-core/pull/nbr)" | replace "nbr" $issueNbr }}
{{- $IssueList = append $IssueList $changeLink }}
{{- end -}}
{{- else }}
{{- $changeLink = "[#nbr](https://github.com/dbt-labs/dbt-core/issues/nbr)" | replace "nbr" $change.Custom.Issue }}
{{- $changes := splitList " " $change.Custom.Issue }}
{{- range $issueNbr := $changes }}
{{- $changeLink := "[#nbr](https://github.com/dbt-labs/dbt-core/issues/nbr)" | replace "nbr" $issueNbr }}
{{- $IssueList = append $IssueList $changeLink }}
{{- end -}}
{{- end }}
{{- /* check if this contributor has other changes associated with them already */}}
{{- if hasKey $contributorDict $author }}
{{- $contributionList := get $contributorDict $author }}
{{- $contributionList = append $contributionList $changeLink }}
{{- $contributionList = concat $contributionList $IssueList }}
{{- $contributorDict := set $contributorDict $author $contributionList }}
{{- else }}
{{- $contributionList := list $changeLink }}
{{- $contributionList := $IssueList }}
{{- $contributorDict := set $contributorDict $author $contributionList }}
{{- end }}
{{- end}}

View File

@@ -61,7 +61,7 @@ body:
label: Environment
description: |
examples:
- **OS**: Ubuntu 20.04
- **OS**: Ubuntu 24.04
- **Python**: 3.9.12 (`python3 --version`)
- **dbt-core**: 1.1.1 (`dbt --version`)
value: |

View File

@@ -55,7 +55,7 @@ body:
label: Environment
description: |
examples:
- **OS**: Ubuntu 20.04
- **OS**: Ubuntu 24.04
- **Python**: 3.9.12 (`python3 --version`)
- **dbt-core (working version)**: 1.1.1 (`dbt --version`)
- **dbt-core (regression version)**: 1.2.0 (`dbt --version`)

20
.github/_README.md vendored
View File

@@ -63,12 +63,12 @@ permissions:
contents: read
pull-requests: write
```
### Secrets
- When to use a [Personal Access Token (PAT)](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) vs the [GITHUB_TOKEN](https://docs.github.com/en/actions/security-guides/automatic-token-authentication) generated for the action?
The `GITHUB_TOKEN` is used by default. In most cases it is sufficient for what you need.
If you expect the workflow to result in a commit to that should retrigger workflows, you will need to use a Personal Access Token for the bot to commit the file. When using the GITHUB_TOKEN, the resulting commit will not trigger another GitHub Actions Workflow run. This is due to limitations set by GitHub. See [the docs](https://docs.github.com/en/actions/security-guides/automatic-token-authentication#using-the-github_token-in-a-workflow) for a more detailed explanation.
For example, we must use a PAT in our workflow to commit a new changelog yaml file for bot PRs. Once the file has been committed to the branch, it should retrigger the check to validate that a changelog exists on the PR. Otherwise, it would stay in a failed state since the check would never retrigger.
@@ -105,7 +105,7 @@ Some triggers of note that we use:
```
# **what?**
# Describe what the action does.
# Describe what the action does.
# **why?**
# Why does this action exist?
@@ -138,7 +138,7 @@ Some triggers of note that we use:
id: fp
run: |
FILEPATH=.changes/unreleased/Dependencies-${{ steps.filename_time.outputs.time }}.yaml
echo "::set-output name=FILEPATH::$FILEPATH"
echo "FILEPATH=$FILEPATH" >> $GITHUB_OUTPUT
```
- Print out all variables you will reference as the first step of a job. This allows for easier debugging. The first job should log all inputs. Subsequent jobs should reference outputs of other jobs, if present.
@@ -158,14 +158,14 @@ Some triggers of note that we use:
echo "The build_script_path: ${{ inputs.build_script_path }}"
echo "The s3_bucket_name: ${{ inputs.s3_bucket_name }}"
echo "The package_test_command: ${{ inputs.package_test_command }}"
# collect all the variables that need to be used in subsequent jobs
- name: Set Variables
id: variables
run: |
echo "::set-output name=important_path::'performance/runner/Cargo.toml'"
echo "::set-output name=release_id::${{github.event.inputs.release_id}}"
echo "::set-output name=open_prs::${{github.event.inputs.open_prs}}"
echo "important_path='performance/runner/Cargo.toml'" >> $GITHUB_OUTPUT
echo "release_id=${{github.event.inputs.release_id}}" >> $GITHUB_OUTPUT
echo "open_prs=${{github.event.inputs.open_prs}}" >> $GITHUB_OUTPUT
job2:
needs: [job1]
@@ -190,7 +190,7 @@ ___
### Actions from the Marketplace
- Dont use external actions for things that can easily be accomplished manually.
- Always read through what an external action does before using it! Often an action in the GitHub Actions Marketplace can be replaced with a few lines in bash. This is much more maintainable (and wont change under us) and clear as to whats actually happening. It also prevents any
- Pin actions _we don't control_ to tags.
- Pin actions _we don't control_ to tags.
### Connecting to AWS
- Authenticate with the aws managed workflow
@@ -208,7 +208,7 @@ ___
```yaml
- name: Copy Artifacts from S3 via CLI
run: aws s3 cp ${{ env.s3_bucket }} . --recursive
run: aws s3 cp ${{ env.s3_bucket }} . --recursive
```
### Testing

View File

@@ -13,7 +13,7 @@ jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: actions/checkout@v4
- name: Wrangle latest tag
id: is_latest
uses: ./.github/actions/latest-wrangler

View File

@@ -28,11 +28,12 @@ if __name__ == "__main__":
if package_request.status_code == 404:
if halt_on_missing:
sys.exit(1)
else:
# everything is the latest if the package doesn't exist
print(f"::set-output name=latest::{True}")
print(f"::set-output name=minor_latest::{True}")
sys.exit(0)
# everything is the latest if the package doesn't exist
github_output = os.environ.get("GITHUB_OUTPUT")
with open(github_output, "at", encoding="utf-8") as gh_output:
gh_output.write("latest=True")
gh_output.write("minor_latest=True")
sys.exit(0)
# TODO: verify package meta is "correct"
# https://github.com/dbt-labs/dbt-core/issues/4640
@@ -91,5 +92,7 @@ if __name__ == "__main__":
latest = is_latest(pre_rel, new_version, current_latest)
minor_latest = is_latest(pre_rel, new_version, current_minor_latest)
print(f"::set-output name=latest::{latest}")
print(f"::set-output name=minor_latest::{minor_latest}")
github_output = os.environ.get("GITHUB_OUTPUT")
with open(github_output, "at", encoding="utf-8") as gh_output:
gh_output.write(f"latest={latest}")
gh_output.write(f"minor_latest={minor_latest}")

View File

@@ -1,10 +0,0 @@
name: "Set up postgres (linux)"
description: "Set up postgres service on linux vm for dbt integration tests"
runs:
using: "composite"
steps:
- shell: bash
run: |
sudo systemctl start postgresql.service
pg_isready
sudo -u postgres bash ${{ github.action_path }}/setup_db.sh

View File

@@ -1 +0,0 @@
../../../test/setup_db.sh

View File

@@ -1,24 +0,0 @@
name: "Set up postgres (macos)"
description: "Set up postgres service on macos vm for dbt integration tests"
runs:
using: "composite"
steps:
- shell: bash
run: |
brew services start postgresql
echo "Check PostgreSQL service is running"
i=10
COMMAND='pg_isready'
while [ $i -gt -1 ]; do
if [ $i == 0 ]; then
echo "PostgreSQL service not ready, all attempts exhausted"
exit 1
fi
echo "Check PostgreSQL service status"
eval $COMMAND && break
echo "PostgreSQL service not ready, wait 10 more sec, attempts left: $i"
sleep 10
((i--))
done
createuser -s postgres
bash ${{ github.action_path }}/setup_db.sh

View File

@@ -1 +0,0 @@
../../../test/setup_db.sh

View File

@@ -42,12 +42,12 @@ jobs:
steps:
- name: Check out the repository
uses: actions/checkout@v2
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4.3.0
uses: actions/setup-python@v5
with:
python-version: '3.8'
python-version: '3.9'
- name: Install python dependencies
run: |
@@ -73,7 +73,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
python-version: [ "3.9", "3.10", "3.11" ]
env:
TOXENV: "unit"
@@ -81,10 +81,10 @@ jobs:
steps:
- name: Check out the repository
uses: actions/checkout@v2
uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4.3.0
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
@@ -95,21 +95,27 @@ jobs:
python -m pip install tox
tox --version
- name: Run tox
run: tox
- name: Run unit tests
uses: nick-fields/retry@v3
with:
timeout_minutes: 10
max_attempts: 3
command: tox -e unit
- name: Get current date
if: always()
id: date
run: echo "::set-output name=date::$(date +'%Y-%m-%dT%H_%M_%S')" #no colons allowed for artifacts
run: |
CURRENT_DATE=$(date +'%Y-%m-%dT%H_%M_%S') # no colons allowed for artifacts
echo "date=$CURRENT_DATE" >> $GITHUB_OUTPUT
- uses: actions/upload-artifact@v2
- uses: actions/upload-artifact@v4
if: always()
with:
name: unit_results_${{ matrix.python-version }}-${{ steps.date.outputs.date }}.csv
path: unit_results.csv
integration:
integration-postgres:
name: integration test / python ${{ matrix.python-version }} / ${{ matrix.os }}
runs-on: ${{ matrix.os }}
@@ -118,13 +124,8 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
os: [ubuntu-20.04]
include:
- python-version: 3.8
os: windows-latest
- python-version: 3.8
os: macos-latest
python-version: [ "3.9", "3.10", "3.11" ]
os: [ubuntu-latest]
env:
TOXENV: integration
@@ -133,23 +134,123 @@ jobs:
DBT_TEST_USER_1: dbt_test_user_1
DBT_TEST_USER_2: dbt_test_user_2
DBT_TEST_USER_3: dbt_test_user_3
DD_CIVISIBILITY_AGENTLESS_ENABLED: true
DD_API_KEY: ${{ secrets.DATADOG_API_KEY }}
DD_SITE: datadoghq.com
DD_ENV: ci
DD_SERVICE: ${{ github.event.repository.name }}
services:
# Label used to access the service container
postgres:
# Docker Hub image
image: postgres
# Provide the password for postgres
env:
POSTGRES_PASSWORD: password
POSTGRES_USER: postgres
# Set health checks to wait until postgres has started
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
steps:
- name: Check out the repository
uses: actions/checkout@v2
uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4.3.0
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Set up postgres (linux)
if: runner.os == 'Linux'
uses: ./.github/actions/setup-postgres-linux
- name: Run postgres setup script
run: |
./test/setup_db.sh
env:
PGHOST: localhost
PGPORT: 5432
PGPASSWORD: password
- name: Install python tools
run: |
python -m pip install --user --upgrade pip
python -m pip --version
python -m pip install tox
tox --version
- name: Run integration tests
uses: nick-fields/retry@v3
with:
timeout_minutes: 90
max_attempts: 3
command: tox -- --ddtrace
- name: Get current date
if: always()
id: date
run: |
CURRENT_DATE=$(date +'%Y-%m-%dT%H_%M_%S') # no colons allowed for artifacts
echo "date=$CURRENT_DATE" >> $GITHUB_OUTPUT
- uses: actions/upload-artifact@v4
if: always()
with:
name: logs_${{ matrix.python-version }}_${{ matrix.os }}_${{ steps.date.outputs.date }}
path: ./logs
- uses: actions/upload-artifact@v4
if: always()
with:
name: integration_results_${{ matrix.python-version }}_${{ matrix.os }}_${{ steps.date.outputs.date }}.csv
path: integration_results.csv
integration-mac-windows:
name: integration test / python ${{ matrix.python-version }} / ${{ matrix.os }}
runs-on: ${{ matrix.os }}
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
include:
- python-version: 3.9
os: windows-latest
- python-version: 3.9
os: macos-14
env:
TOXENV: integration
PYTEST_ADDOPTS: "-v --color=yes -n4 --csv integration_results.csv"
DBT_INVOCATION_ENV: github-actions
DBT_TEST_USER_1: dbt_test_user_1
DBT_TEST_USER_2: dbt_test_user_2
DBT_TEST_USER_3: dbt_test_user_3
DD_CIVISIBILITY_AGENTLESS_ENABLED: true
DD_API_KEY: ${{ secrets.DATADOG_API_KEY }}
DD_SITE: datadoghq.com
DD_ENV: ci
DD_SERVICE: ${{ github.event.repository.name }}
steps:
- name: Check out the repository
uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Set up postgres (macos)
if: runner.os == 'macOS'
uses: ./.github/actions/setup-postgres-macos
uses: nick-fields/retry@v3
with:
timeout_minutes: 10
max_attempts: 3
command: ./test/setup_db.sh
- name: Set up postgres (windows)
if: runner.os == 'Windows'
@@ -162,21 +263,27 @@ jobs:
python -m pip install tox
tox --version
- name: Run tests
run: tox
- name: Run integration tests
uses: nick-fields/retry@v3
with:
timeout_minutes: 90
max_attempts: 3
command: tox -- --ddtrace
- name: Get current date
if: always()
id: date
run: echo "::set-output name=date::$(date +'%Y_%m_%dT%H_%M_%S')" #no colons allowed for artifacts
run: |
CURRENT_DATE=$(date +'%Y-%m-%dT%H_%M_%S') # no colons allowed for artifacts
echo "date=$CURRENT_DATE" >> $GITHUB_OUTPUT
- uses: actions/upload-artifact@v2
- uses: actions/upload-artifact@v4
if: always()
with:
name: logs_${{ matrix.python-version }}_${{ matrix.os }}_${{ steps.date.outputs.date }}
path: ./logs
- uses: actions/upload-artifact@v2
- uses: actions/upload-artifact@v4
if: always()
with:
name: integration_results_${{ matrix.python-version }}_${{ matrix.os }}_${{ steps.date.outputs.date }}.csv
@@ -189,12 +296,12 @@ jobs:
steps:
- name: Check out the repository
uses: actions/checkout@v2
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4.3.0
uses: actions/setup-python@v5
with:
python-version: '3.8'
python-version: '3.9'
- name: Install python dependencies
run: |
@@ -227,7 +334,7 @@ jobs:
- name: Install source distributions
# ignore dbt-1.0.0, which intentionally raises an error when installed from source
run: |
find ./dist/dbt-[a-z]*.gz -maxdepth 1 -type f | xargs python -m pip install --force-reinstall --find-links=dist/
find ./dist/*.gz -maxdepth 1 -type f | xargs python -m pip install --force-reinstall --find-links=dist/
- name: Check source distributions
run: |

109
.github/workflows/nightly-release.yml vendored Normal file
View File

@@ -0,0 +1,109 @@
# **what?**
# Nightly releases to GitHub and PyPI. This workflow produces the following outcome:
# - generate and validate data for night release (commit SHA, version number, release branch);
# - pass data to release workflow;
# - night release will be pushed to GitHub as a draft release;
# - night build will be pushed to test PyPI;
#
# **why?**
# Ensure an automated and tested release process for nightly builds
#
# **when?**
# This workflow runs on schedule or can be run manually on demand.
name: Nightly Test Release to GitHub and PyPI
on:
workflow_dispatch: # for manual triggering
schedule:
- cron: 0 9 * * *
permissions:
contents: write # this is the permission that allows creating a new release
defaults:
run:
shell: bash
env:
RELEASE_BRANCH: "main"
jobs:
aggregate-release-data:
runs-on: ubuntu-latest
outputs:
commit_sha: ${{ steps.resolve-commit-sha.outputs.release_commit }}
version_number: ${{ steps.nightly-release-version.outputs.number }}
release_branch: ${{ steps.release-branch.outputs.name }}
steps:
- name: "Checkout ${{ github.repository }} Branch ${{ env.RELEASE_BRANCH }}"
uses: actions/checkout@v4
with:
ref: ${{ env.RELEASE_BRANCH }}
- name: "Resolve Commit To Release"
id: resolve-commit-sha
run: |
commit_sha=$(git rev-parse HEAD)
echo "release_commit=$commit_sha" >> $GITHUB_OUTPUT
- name: "Get Current Version Number"
id: version-number-sources
run: |
current_version=`awk -F"current_version = " '{print $2}' .bumpversion.cfg | tr '\n' ' '`
echo "current_version=$current_version" >> $GITHUB_OUTPUT
- name: "Audit Version And Parse Into Parts"
id: semver
uses: dbt-labs/actions/parse-semver@v1.1.0
with:
version: ${{ steps.version-number-sources.outputs.current_version }}
- name: "Get Current Date"
id: current-date
run: echo "date=$(date +'%m%d%Y')" >> $GITHUB_OUTPUT
- name: "Generate Nightly Release Version Number"
id: nightly-release-version
run: |
number="${{ steps.semver.outputs.version }}.dev${{ steps.current-date.outputs.date }}+nightly"
echo "number=$number" >> $GITHUB_OUTPUT
- name: "Audit Nightly Release Version And Parse Into Parts"
uses: dbt-labs/actions/parse-semver@v1.1.0
with:
version: ${{ steps.nightly-release-version.outputs.number }}
- name: "Set Release Branch"
id: release-branch
run: |
echo "name=${{ env.RELEASE_BRANCH }}" >> $GITHUB_OUTPUT
log-outputs-aggregate-release-data:
runs-on: ubuntu-latest
needs: [aggregate-release-data]
steps:
- name: "[DEBUG] Log Outputs"
run: |
echo commit_sha : ${{ needs.aggregate-release-data.outputs.commit_sha }}
echo version_number: ${{ needs.aggregate-release-data.outputs.version_number }}
echo release_branch: ${{ needs.aggregate-release-data.outputs.release_branch }}
release-github-pypi:
needs: [aggregate-release-data]
uses: ./.github/workflows/release.yml
with:
sha: ${{ needs.aggregate-release-data.outputs.commit_sha }}
target_branch: ${{ needs.aggregate-release-data.outputs.release-branch }}
version_number: ${{ needs.aggregate-release-data.outputs.version_number }}
build_script_path: "scripts/build-dist.sh"
env_setup_script_path: "scripts/env-setup.sh"
s3_bucket_name: "core-team-artifacts"
package_test_command: "dbt --version"
test_run: true
nightly_release: true
secrets: inherit

Some files were not shown because too many files have changed in this diff Show More