Compare commits

...

9 Commits

Author SHA1 Message Date
github-actions[bot]
cbd4570c67 Bumping version to 1.1.0 (#5186)
* Bumping version to 1.1.0

* Updating changelog

Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: Leah Antkiewicz <leah.antkiewicz@fishtownanalytics.com>
2022-04-28 12:54:30 -04:00
github-actions[bot]
5ae33b90bf Bumping version to 1.1.0rc3 (#5166)
* Bumping version to 1.1.0rc3

* Update changelog

* Fixing spacing issue

* Removing more whitespace

Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: Leah Antkiewicz <leah.antkiewicz@fishtownanalytics.com>
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com>
2022-04-26 15:25:39 -04:00
github-actions[bot]
37d78338c2 [Backport 1.1.latest] Use yaml renderer (with target context) for rendering selectors (#5161)
* Use yaml renderer (with target context) for rendering selectors (#5136)

* Use yaml renderer (with target context) for rendering selectors

* Changie

* Convert cli_vars tests

* Add test for var in profiles

* Add test for cli vars in packages

* Add test for vars in selectors

(cherry picked from commit 1f898c859a)

* Tweak cli_vars_in_packages test to do a run and build an adapter

Co-authored-by: Gerda Shank <gerda@dbtlabs.com>
2022-04-26 14:13:16 -04:00
github-actions[bot]
698420b420 Even more scrubbing (#5152) (#5156)
* Even more scrubbing

* Changelog entry

* Even more

* remove reduendent scrub

* remove reduendent scrub

* fix encoding issue

* keep scrubbed log in args

Co-authored-by: Chenyu Li <chenyu.li@dbtlabs.com>
(cherry picked from commit ce0bcc08a6)

Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com>
2022-04-26 10:59:07 -06:00
Emily Rockman
e23f0e0747 fix retry logic failures (#5137) (#5148)
* fix retry logic failures

* changelog

* add tests to make sure data is getting where it needs to

* rename file

* remove duplicate file

(cherry picked from commit 31a3f2bdee)
2022-04-25 10:45:35 -05:00
github-actions[bot]
106da05db7 Bumping version to 1.1.0rc2 (#5129)
* Bumping version to 1.1.0rc2

* release changelog

* fix trailing space

Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: Chenyu Li <chenyu.li@dbtlabs.com>
2022-04-21 09:52:08 -06:00
Chenyu Li
09a396a731 Restore ability to utilize updated_at for check_cols snapshots (#5077) (#5126)
* Restore ability to configure and utilize `updated_at` for snapshots using the check_cols strategy

* Changelog entry

* Optional comparison of column names starting with `dbt_`

* Functional test for check cols snapshots using `updated_at`

* Comments to explain the test implementation

(cherry picked from commit d09459c980)

Co-authored-by: Doug Beatty <44704949+dbeatty10@users.noreply.github.com>
2022-04-21 09:06:04 -06:00
Emily Rockman
9c233f27ab remove commas from changelog yaml (#5113) 2022-04-19 19:57:14 -05:00
github-actions[bot]
a09bc28768 Bumping version to 1.1.0rc1 (#5044)
* Bumping version to 1.1.0rc1

* Adding changelog

* Updating adapter versions for Docker

* Remove extra spaces

Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: Leah Antkiewicz <leah.antkiewicz@fishtownanalytics.com>
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com>
2022-04-12 17:10:57 -04:00
98 changed files with 690 additions and 753 deletions

View File

@@ -1,5 +1,5 @@
[bumpversion]
current_version = 1.1.0b1
current_version = 1.1.0
parse = (?P<major>\d+)
\.(?P<minor>\d+)
\.(?P<patch>\d+)

View File

@@ -1,50 +0,0 @@
## dbt-core 1.1.0-b1 - March 22, 2022
### Breaking Changes
- **Relevant to maintainers of adapter plugins _only_:** The abstractmethods `get_response` and `execute` now only return a `connection.AdapterReponse` in type hints. (Previously, they could return a string.) We encourage you to update your methods to return an object of class `AdapterResponse`, or implement a subclass specific to your adapter ([#4499](https://github.com/dbt-labs/dbt-core/issues/4499), [#4869](https://github.com/dbt-labs/dbt-core/pull/4869))
### Features
- Change behaviour of `non_null` test so that it only `select`s all columns if `--store-failures` is enabled. ([#4769](https://github.com/dbt-labs/dbt-core/issues/4769), [#4777](https://github.com/dbt-labs/dbt-core/pull/4777))
- Testing framework for dbt adapter testing ([#4730](https://github.com/dbt-labs/dbt-core/issues/4730), [#4846](https://github.com/dbt-labs/dbt-core/pull/4846))
- Allow unique key to take a list implementation for postgres/redshift ([#4738](https://github.com/dbt-labs/dbt-core/issues/4738), [#4858](https://github.com/dbt-labs/dbt-core/pull/4858))
- Added Support for Semantic Versioning ([#4453](https://github.com/dbt-labs/dbt-core/issues/4453), [#4644](https://github.com/dbt-labs/dbt-core/pull/4644))
- New Dockerfile to support specific db adapters and platforms. See docker/README.md for details ([#4495](https://github.com/dbt-labs/dbt-core/issues/4495), [#4487](https://github.com/dbt-labs/dbt-core/pull/4487))
- Allow unique_key to take a list ([#2479](https://github.com/dbt-labs/dbt-core/issues/2479), [#4618](https://github.com/dbt-labs/dbt-core/pull/4618))
- Add `--quiet` global flag and `print` Jinja function ([#3451](https://github.com/dbt-labs/dbt-core/issues/3451), [#4701](https://github.com/dbt-labs/dbt-core/pull/4701))
- Add space before justification periods ([#4737](https://github.com/dbt-labs/dbt-core/issues/4737), [#4744](https://github.com/dbt-labs/dbt-core/pull/4744))
### Fixes
- Fix bug causing empty node level meta, snapshot config errors ([#4459](https://github.com/dbt-labs/dbt-core/issues/4459), [#4726](https://github.com/dbt-labs/dbt-core/pull/4726))
- Catch all Requests Exceptions on deps install to attempt retries. Also log the exceptions hit. ([#4849](https://github.com/dbt-labs/dbt-core/issues/4849), [#4865](https://github.com/dbt-labs/dbt-core/pull/4865))
- Use cli_vars instead of context to create package and selector renderers ([#4876](https://github.com/dbt-labs/dbt-core/issues/4876), [#4878](https://github.com/dbt-labs/dbt-core/pull/4878))
- depend on new dbt-extractor version with fixed github links ([#4891](https://github.com/dbt-labs/dbt-core/issues/4891), [#4890](https://github.com/dbt-labs/dbt-core/pull/4890))
- Update bumpervsion config to stop looking for missing setup.py ([#-1](https://github.com/dbt-labs/dbt-core/issues/-1), [#4896](https://github.com/dbt-labs/dbt-core/pull/4896))
- Corrected permissions settings for docker release workflow ([#4902](https://github.com/dbt-labs/dbt-core/issues/4902), [#4903](https://github.com/dbt-labs/dbt-core/pull/4903))
- User wasn't asked for permission to overwite a profile entry when running init inside an existing project ([#4375](https://github.com/dbt-labs/dbt-core/issues/4375), [#4447](https://github.com/dbt-labs/dbt-core/pull/4447))
- Add project name validation to `dbt init` ([#4490](https://github.com/dbt-labs/dbt-core/issues/4490), [#4536](https://github.com/dbt-labs/dbt-core/pull/4536))
- Allow override of string and numeric types for adapters. ([#4603](https://github.com/dbt-labs/dbt-core/issues/4603), [#4604](https://github.com/dbt-labs/dbt-core/pull/4604))
- A change in secret environment variables won't trigger a full reparse ([#4650](https://github.com/dbt-labs/dbt-core/issues/4650), [#4665](https://github.com/dbt-labs/dbt-core/pull/4665))
- Fix misspellings and typos in docstrings ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#4545](https://github.com/dbt-labs/dbt-core/pull/4545))
### Under the Hood
- Automate changelog generation with changie ([#4652](https://github.com/dbt-labs/dbt-core/issues/4652), [#4743](https://github.com/dbt-labs/dbt-core/pull/4743))
- Fix broken links for changelog generation and tweak GHA to only post a comment once when changelog entry is missing. ([#4848](https://github.com/dbt-labs/dbt-core/issues/4848), [#4857](https://github.com/dbt-labs/dbt-core/pull/4857))
- Add support for Python 3.10 ([#4562](https://github.com/dbt-labs/dbt-core/issues/4562), [#4866](https://github.com/dbt-labs/dbt-core/pull/4866))
- Enable more dialects to snapshot sources with added columns, even those that don't support boolean datatypes ([#4488](https://github.com/dbt-labs/dbt-core/issues/4488), [#4871](https://github.com/dbt-labs/dbt-core/pull/4871))
- Testing cleanup ([#3648](https://github.com/dbt-labs/dbt-core/issues/3648), [#4509](https://github.com/dbt-labs/dbt-core/pull/4509))
- Clean up test deprecation warnings ([#3988](https://github.com/dbt-labs/dbt-core/issues/3988), [#4556](https://github.com/dbt-labs/dbt-core/pull/4556))
- Use mashumaro for serialization in event logging ([#4504](https://github.com/dbt-labs/dbt-core/issues/4504), [#4505](https://github.com/dbt-labs/dbt-core/pull/4505))
- Drop support for Python 3.7.0 + 3.7.1 ([#4584](https://github.com/dbt-labs/dbt-core/issues/4584), [#4585](https://github.com/dbt-labs/dbt-core/pull/4585))
- Drop support for Python <3.7.2 ([#4584](https://github.com/dbt-labs/dbt-core/issues/4584), [#4643](https://github.com/dbt-labs/dbt-core/pull/4643))
- Re-format codebase (except tests) using pre-commit hooks ([#3195](https://github.com/dbt-labs/dbt-core/issues/3195), [#4697](https://github.com/dbt-labs/dbt-core/pull/4697))
- Add deps module README ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#4686](https://github.com/dbt-labs/dbt-core/pull/4686))
- Initial conversion of tests to pytest ([#4690](https://github.com/dbt-labs/dbt-core/issues/4690), [#4691](https://github.com/dbt-labs/dbt-core/pull/4691))
- Fix errors in Windows for tests/functions ([#4782](https://github.com/dbt-labs/dbt-core/issues/4782), [#4767](https://github.com/dbt-labs/dbt-core/pull/4767))
### Docs
- Resolve errors related to operations preventing DAG from generating in the docs. Also patch a spark issue to allow search to filter accurately past the missing columns. ([#4578](https://github.com/dbt-labs/dbt-core/issues/4578), [#4763](https://github.com/dbt-labs/dbt-core/pull/4763))
### Contributors
- [@NiallRees](https://github.com/NiallRees) ([#4447](https://github.com/dbt-labs/dbt-core/pull/4447))
- [@alswang18](https://github.com/alswang18) ([#4644](https://github.com/dbt-labs/dbt-core/pull/4644))
- [@amirkdv](https://github.com/amirkdv) ([#4536](https://github.com/dbt-labs/dbt-core/pull/4536))
- [@ehmartens](https://github.com/ehmartens) ([#4701](https://github.com/dbt-labs/dbt-core/pull/4701))
- [@joellabes](https://github.com/joellabes) ([#4744](https://github.com/dbt-labs/dbt-core/pull/4744))
- [@kazanzhy](https://github.com/kazanzhy) ([#4545](https://github.com/dbt-labs/dbt-core/pull/4545))
- [@mdesmet](https://github.com/mdesmet) ([#4604](https://github.com/dbt-labs/dbt-core/pull/4604))
- [@versusfacit](https://github.com/versusfacit) ([#4869](https://github.com/dbt-labs/dbt-core/pull/4869))
- [@willbowditch](https://github.com/willbowditch) ([#4777](https://github.com/dbt-labs/dbt-core/pull/4777))

96
.changes/1.1.0.md Normal file
View File

@@ -0,0 +1,96 @@
## dbt-core 1.1.0 - April 28, 2022
### Breaking Changes
- **Relevant to maintainers of adapter plugins _only_:** The abstractmethods `get_response` and `execute` now only return a `connection.AdapterReponse` in type hints. (Previously, they could return a string.) We encourage you to update your methods to return an object of class `AdapterResponse`, or implement a subclass specific to your adapter
([#4499](https://github.com/dbt-labs/dbt-core/issues/4499), [#4869](https://github.com/dbt-labs/dbt-core/pull/4869))
- For adapter plugin maintainers only: Internal adapter methods `set_relations_cache` + `_relations_cache_for_schemas` each take an additional argument, for use with experimental `CACHE_SELECTED_ONLY` config ([#4688](https://github.com/dbt-labs/dbt-core/issues/4688), [#4860](https://github.com/dbt-labs/dbt-core/pull/4860))
### Features
- Change behaviour of `non_null` test so that it only `select`s all columns if `--store-failures` is enabled. ([#4769](https://github.com/dbt-labs/dbt-core/issues/4769), [#4777](https://github.com/dbt-labs/dbt-core/pull/4777))
- Testing framework for dbt adapter testing ([#4730](https://github.com/dbt-labs/dbt-core/issues/4730), [#4846](https://github.com/dbt-labs/dbt-core/pull/4846))
- Allow unique key to take a list implementation for postgres/redshift ([#4738](https://github.com/dbt-labs/dbt-core/issues/4738), [#4858](https://github.com/dbt-labs/dbt-core/pull/4858))
- Add `--cache_selected_only` flag to cache schema object of selected models only. ([#4688](https://github.com/dbt-labs/dbt-core/issues/4688), [#4860](https://github.com/dbt-labs/dbt-core/pull/4860))
- Support custom names for generic tests ([#3348](https://github.com/dbt-labs/dbt-core/issues/3348), [#4898](https://github.com/dbt-labs/dbt-core/pull/4898))
- Added Support for Semantic Versioning ([#4453](https://github.com/dbt-labs/dbt-core/issues/4453), [#4644](https://github.com/dbt-labs/dbt-core/pull/4644))
- New Dockerfile to support specific db adapters and platforms. See docker/README.md for details ([#4495](https://github.com/dbt-labs/dbt-core/issues/4495), [#4487](https://github.com/dbt-labs/dbt-core/pull/4487))
- Allow unique_key to take a list ([#2479](https://github.com/dbt-labs/dbt-core/issues/2479), [#4618](https://github.com/dbt-labs/dbt-core/pull/4618))
- Add `--quiet` global flag and `print` Jinja function ([#3451](https://github.com/dbt-labs/dbt-core/issues/3451), [#4701](https://github.com/dbt-labs/dbt-core/pull/4701))
- Add space before justification periods ([#4737](https://github.com/dbt-labs/dbt-core/issues/4737), [#4744](https://github.com/dbt-labs/dbt-core/pull/4744))
- Enable dbt jobs to run downstream models based on fresher sources. Compare the source freshness results between previous and current state. If any source is fresher and/or new in current vs. previous state, dbt will run and test the downstream models in scope. Example command: `dbt build --select source_status:fresher+` ([#4050](https://github.com/dbt-labs/dbt-core/issues/4050), [#4256](https://github.com/dbt-labs/dbt-core/pull/4256))
- converting unique key as list tests to new pytest format ([#4882](https://github.com/dbt-labs/dbt-core/issues/4882), [#4958](https://github.com/dbt-labs/dbt-core/pull/4958))
- Add a variable called selected_resources in the Jinja context containing a list of all the resources matching the nodes for the --select, --exclude and/or --selector parameters. ([#3471](https://github.com/dbt-labs/dbt-core/issues/3471), [#5001](https://github.com/dbt-labs/dbt-core/pull/5001))
- Support the DO_NOT_TRACK environment variable from the consoledonottrack.com initiative ([#3540](https://github.com/dbt-labs/dbt-core/issues/3540), [#5000](https://github.com/dbt-labs/dbt-core/pull/5000))
- Add `--no-print` global flag ([#4710](https://github.com/dbt-labs/dbt-core/issues/4710), [#4854](https://github.com/dbt-labs/dbt-core/pull/4854))
- add enabled as a source config ([#3662](https://github.com/dbt-labs/dbt-core/issues/3662), [#5008](https://github.com/dbt-labs/dbt-core/pull/5008))
### Fixes
- Fix bug causing empty node level meta, snapshot config errors ([#4459](https://github.com/dbt-labs/dbt-core/issues/4459), [#4726](https://github.com/dbt-labs/dbt-core/pull/4726))
- Inconsistent timestamps between inserted/updated and deleted rows in snapshots ([#4347](https://github.com/dbt-labs/dbt-core/issues/4347), [#4513](https://github.com/dbt-labs/dbt-core/pull/4513))
- Catch all Requests Exceptions on deps install to attempt retries. Also log the exceptions hit. ([#4849](https://github.com/dbt-labs/dbt-core/issues/4849), [#4865](https://github.com/dbt-labs/dbt-core/pull/4865))
- Use cli_vars instead of context to create package and selector renderers ([#4876](https://github.com/dbt-labs/dbt-core/issues/4876), [#4878](https://github.com/dbt-labs/dbt-core/pull/4878))
- depend on new dbt-extractor version with fixed github links ([#4891](https://github.com/dbt-labs/dbt-core/issues/4891), [#4890](https://github.com/dbt-labs/dbt-core/pull/4890))
- Update bumpervsion config to stop looking for missing setup.py ([#-1](https://github.com/dbt-labs/dbt-core/issues/-1), [#4896](https://github.com/dbt-labs/dbt-core/pull/4896))
- Corrected permissions settings for docker release workflow ([#4902](https://github.com/dbt-labs/dbt-core/issues/4902), [#4903](https://github.com/dbt-labs/dbt-core/pull/4903))
- User wasn't asked for permission to overwite a profile entry when running init inside an existing project ([#4375](https://github.com/dbt-labs/dbt-core/issues/4375), [#4447](https://github.com/dbt-labs/dbt-core/pull/4447))
- Add project name validation to `dbt init` ([#4490](https://github.com/dbt-labs/dbt-core/issues/4490), [#4536](https://github.com/dbt-labs/dbt-core/pull/4536))
- Allow override of string and numeric types for adapters. ([#4603](https://github.com/dbt-labs/dbt-core/issues/4603), [#4604](https://github.com/dbt-labs/dbt-core/pull/4604))
- A change in secret environment variables won't trigger a full reparse ([#4650](https://github.com/dbt-labs/dbt-core/issues/4650), [#4665](https://github.com/dbt-labs/dbt-core/pull/4665))
- Fix misspellings and typos in docstrings ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#4545](https://github.com/dbt-labs/dbt-core/pull/4545))
- Catch more cases to retry package retrieval for deps pointing to the hub. Also start to cache the package requests. ([#4849](https://github.com/dbt-labs/dbt-core/issues/4849), [#4982](https://github.com/dbt-labs/dbt-core/pull/4982))
- Make the warning message for a full event deque more descriptive ([#4962](https://github.com/dbt-labs/dbt-core/issues/4962), [#5011](https://github.com/dbt-labs/dbt-core/pull/5011))
- Fix hard delete snapshot test ([#4916](https://github.com/dbt-labs/dbt-core/issues/4916), [#5020](https://github.com/dbt-labs/dbt-core/pull/5020))
- Restore ability to utilize `updated_at` for check_cols snapshots ([#5076](https://github.com/dbt-labs/dbt-core/issues/5076), [#5077](https://github.com/dbt-labs/dbt-core/pull/5077))
- Use yaml renderer (with target context) for rendering selectors ([#5131](https://github.com/dbt-labs/dbt-core/issues/5131), [#5136](https://github.com/dbt-labs/dbt-core/pull/5136))
- Fix retry logic to return values after initial try ([#5023](https://github.com/dbt-labs/dbt-core/issues/5023), [#5137](https://github.com/dbt-labs/dbt-core/pull/5137))
- Scrub secret env vars from CommandError in exception stacktrace ([#5151](https://github.com/dbt-labs/dbt-core/issues/5151), [#5152](https://github.com/dbt-labs/dbt-core/pull/5152))
### Docs
- Resolve errors related to operations preventing DAG from generating in the docs. Also patch a spark issue to allow search to filter accurately past the missing columns. ([#4578](https://github.com/dbt-labs/dbt-core/issues/4578), [#4763](https://github.com/dbt-labs/dbt-core/pull/4763))
- Fixed capitalization in UI for exposures of `type: ml` ([#4984](https://github.com/dbt-labs/dbt-core/issues/4984), [#4995](https://github.com/dbt-labs/dbt-core/pull/4995))
- List packages and tags in alphabetical order ([#4984](https://github.com/dbt-labs/dbt-core/issues/4984), [#4995](https://github.com/dbt-labs/dbt-core/pull/4995))
- Bump jekyll from 3.8.7 to 3.9.0 ([#4984](https://github.com/dbt-labs/dbt-core/issues/4984), [#4995](https://github.com/dbt-labs/dbt-core/pull/4995))
- Updated docker README to reflect necessity of using BuildKit ([#4990](https://github.com/dbt-labs/dbt-core/issues/4990), [#5018](https://github.com/dbt-labs/dbt-core/pull/5018))
### Under the Hood
- Automate changelog generation with changie ([#4652](https://github.com/dbt-labs/dbt-core/issues/4652), [#4743](https://github.com/dbt-labs/dbt-core/pull/4743))
- add performance regression testing runner without orchestration ([#4021](https://github.com/dbt-labs/dbt-core/issues/4021), [#4602](https://github.com/dbt-labs/dbt-core/pull/4602))
- Fix broken links for changelog generation and tweak GHA to only post a comment once when changelog entry is missing. ([#4848](https://github.com/dbt-labs/dbt-core/issues/4848), [#4857](https://github.com/dbt-labs/dbt-core/pull/4857))
- Add support for Python 3.10 ([#4562](https://github.com/dbt-labs/dbt-core/issues/4562), [#4866](https://github.com/dbt-labs/dbt-core/pull/4866))
- Enable more dialects to snapshot sources with added columns, even those that don't support boolean datatypes ([#4488](https://github.com/dbt-labs/dbt-core/issues/4488), [#4871](https://github.com/dbt-labs/dbt-core/pull/4871))
- Add Graph Compilation and Adapter Cache tracking ([#4625](https://github.com/dbt-labs/dbt-core/issues/4625), [#4912](https://github.com/dbt-labs/dbt-core/pull/4912))
- Testing cleanup ([#3648](https://github.com/dbt-labs/dbt-core/issues/3648), [#4509](https://github.com/dbt-labs/dbt-core/pull/4509))
- Clean up test deprecation warnings ([#3988](https://github.com/dbt-labs/dbt-core/issues/3988), [#4556](https://github.com/dbt-labs/dbt-core/pull/4556))
- Use mashumaro for serialization in event logging ([#4504](https://github.com/dbt-labs/dbt-core/issues/4504), [#4505](https://github.com/dbt-labs/dbt-core/pull/4505))
- Drop support for Python 3.7.0 + 3.7.1 ([#4584](https://github.com/dbt-labs/dbt-core/issues/4584), [#4585](https://github.com/dbt-labs/dbt-core/pull/4585))
- Drop support for Python <3.7.2 ([#4584](https://github.com/dbt-labs/dbt-core/issues/4584), [#4643](https://github.com/dbt-labs/dbt-core/pull/4643))
- Re-format codebase (except tests) using pre-commit hooks ([#3195](https://github.com/dbt-labs/dbt-core/issues/3195), [#4697](https://github.com/dbt-labs/dbt-core/pull/4697))
- Add deps module README ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#4686](https://github.com/dbt-labs/dbt-core/pull/4686))
- Initial conversion of tests to pytest ([#4690](https://github.com/dbt-labs/dbt-core/issues/4690), [#4691](https://github.com/dbt-labs/dbt-core/pull/4691))
- Fix errors in Windows for tests/functions ([#4782](https://github.com/dbt-labs/dbt-core/issues/4782), [#4767](https://github.com/dbt-labs/dbt-core/pull/4767))
- Create a dbt.tests.adapter release when releasing dbt and postgres ([#4812](https://github.com/dbt-labs/dbt-core/issues/4812), [#4948](https://github.com/dbt-labs/dbt-core/pull/4948))
- update docker image to use python 3.10.3 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#4963](https://github.com/dbt-labs/dbt-core/pull/4963))
- updates black to 22.3.0 which fixes dependency incompatibility when running with precommit. ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#4972](https://github.com/dbt-labs/dbt-core/pull/4972))
- Adds config util for ad-hoc creation of project objs or dicts ([#4808](https://github.com/dbt-labs/dbt-core/issues/4808), [#4981](https://github.com/dbt-labs/dbt-core/pull/4981))
- Remove TableComparison and convert existing calls to use dbt.tests.util ([#4778](https://github.com/dbt-labs/dbt-core/issues/4778), [#4986](https://github.com/dbt-labs/dbt-core/pull/4986))
- Remove unneeded create_schema in snapshot materialization ([#4742](https://github.com/dbt-labs/dbt-core/issues/4742), [#4993](https://github.com/dbt-labs/dbt-core/pull/4993))
- Added .git-blame-ignore-revs file to mask re-formmating commits from git blame ([#5004](https://github.com/dbt-labs/dbt-core/issues/5004), [#5019](https://github.com/dbt-labs/dbt-core/pull/5019))
- Convert version tests to pytest ([#5024](https://github.com/dbt-labs/dbt-core/issues/5024), [#5026](https://github.com/dbt-labs/dbt-core/pull/5026))
- Updating tests and docs to show that we now support Python 3.10 ([#4974](https://github.com/dbt-labs/dbt-core/issues/4974), [#5025](https://github.com/dbt-labs/dbt-core/pull/5025))
- Update --version output and logic ([#4724](https://github.com/dbt-labs/dbt-core/issues/4724), [#5029](https://github.com/dbt-labs/dbt-core/pull/5029))
- ([#5033](https://github.com/dbt-labs/dbt-core/issues/5033), [#5032](https://github.com/dbt-labs/dbt-core/pull/5032))
### Contributors
- [@NiallRees](https://github.com/NiallRees) ([#4447](https://github.com/dbt-labs/dbt-core/pull/4447))
- [@agoblet](https://github.com/agoblet) ([#5000](https://github.com/dbt-labs/dbt-core/pull/5000))
- [@alswang18](https://github.com/alswang18) ([#4644](https://github.com/dbt-labs/dbt-core/pull/4644))
- [@amirkdv](https://github.com/amirkdv) ([#4536](https://github.com/dbt-labs/dbt-core/pull/4536))
- [@anaisvaillant](https://github.com/anaisvaillant) ([#4256](https://github.com/dbt-labs/dbt-core/pull/4256))
- [@b-per](https://github.com/b-per) ([#5001](https://github.com/dbt-labs/dbt-core/pull/5001))
- [@dbeatty10](https://github.com/dbeatty10) ([#5077](https://github.com/dbt-labs/dbt-core/pull/5077))
- [@ehmartens](https://github.com/ehmartens) ([#4701](https://github.com/dbt-labs/dbt-core/pull/4701))
- [@joellabes](https://github.com/joellabes) ([#4744](https://github.com/dbt-labs/dbt-core/pull/4744))
- [@jonstacks](https://github.com/jonstacks) ([#4995](https://github.com/dbt-labs/dbt-core/pull/4995))
- [@kadero](https://github.com/kadero) ([#4513](https://github.com/dbt-labs/dbt-core/pull/4513))
- [@karunpoudel](https://github.com/karunpoudel) ([#4860](https://github.com/dbt-labs/dbt-core/pull/4860), [#4860](https://github.com/dbt-labs/dbt-core/pull/4860))
- [@kazanzhy](https://github.com/kazanzhy) ([#4545](https://github.com/dbt-labs/dbt-core/pull/4545))
- [@matt-winkler](https://github.com/matt-winkler) ([#4256](https://github.com/dbt-labs/dbt-core/pull/4256))
- [@mdesmet](https://github.com/mdesmet) ([#4604](https://github.com/dbt-labs/dbt-core/pull/4604))
- [@pgoslatara](https://github.com/pgoslatara) ([#4995](https://github.com/dbt-labs/dbt-core/pull/4995))
- [@poloaraujo](https://github.com/poloaraujo) ([#4854](https://github.com/dbt-labs/dbt-core/pull/4854))
- [@sungchun12](https://github.com/sungchun12) ([#4256](https://github.com/dbt-labs/dbt-core/pull/4256))
- [@willbowditch](https://github.com/willbowditch) ([#4777](https://github.com/dbt-labs/dbt-core/pull/4777))

View File

@@ -1,11 +0,0 @@
kind: Breaking Changes
body: >
**Relevant to maintainers of adapter plugins _only_:**
The abstractmethods `get_response` and `execute` now only return a `connection.AdapterReponse` in type hints.
(Previously, they could return a string.) We encourage you to update your methods to return an object of
class `AdapterResponse`, or implement a subclass specific to your adapter
time: 2022-03-16T03:13:11.656007-07:00
custom:
Author: versusfacit
Issue: "4499"
PR: "4869"

View File

@@ -1,8 +0,0 @@
kind: Docs
body: Resolve errors related to operations preventing DAG from generating in the docs. Also
patch a spark issue to allow search to filter accurately past the missing columns.
time: 2022-03-07T20:31:05.557064-06:00
custom:
Author: emmyoop
Issue: "4578"
PR: "4763"

View File

@@ -1,7 +0,0 @@
kind: Features
body: Testing framework for dbt adapter testing
time: 2022-03-09T14:26:45.828295-05:00
custom:
Author: gshank
Issue: "4730"
PR: "4846"

View File

@@ -1,8 +0,0 @@
kind: Features
body: Change behaviour of `non_null` test so that it only `select`s all columns if
`--store-failures` is enabled.
time: 2022-03-09T16:38:17.160568Z
custom:
Author: willbowditch
Issue: "4769"
PR: "4777"

View File

@@ -1,7 +0,0 @@
kind: Features
body: Allow unique key to take a list implementation for postgres/redshift
time: 2022-03-14T11:23:41.293726-05:00
custom:
Author: McKnight-42
Issue: "4738"
PR: "4858"

View File

@@ -1,7 +0,0 @@
kind: Features
body: Added Support for Semantic Versioning
time: 2022-03-21T13:59:49.725046-05:00
custom:
Author: alswang18
Issue: "4453"
PR: "4644"

View File

@@ -1,8 +0,0 @@
kind: Features
body: New Dockerfile to support specific db adapters and platforms. See docker/README.md
for details
time: 2022-03-21T14:00:46.670045-05:00
custom:
Author: iknox-fa
Issue: "4495"
PR: "4487"

View File

@@ -1,7 +0,0 @@
kind: Features
body: Allow unique_key to take a list
time: 2022-03-21T14:01:39.918897-05:00
custom:
Author: gshank
Issue: "2479"
PR: "4618"

View File

@@ -1,7 +0,0 @@
kind: Features
body: "Add `--quiet` global flag and `print` Jinja function"
time: 2022-03-21T14:03:06.565157-05:00
custom:
Author: ehmartens
Issue: "3451"
PR: "4701"

View File

@@ -1,7 +0,0 @@
kind: Features
body: Add space before justification periods
time: 2022-03-22T14:48:22.64892-04:00
custom:
Author: joellabes
Issue: "4737"
PR: "4744"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Fix bug causing empty node level meta, snapshot config errors
time: 2022-03-07T20:30:22.624709-06:00
custom:
Author: gshank
Issue: "4459"
PR: "4726"

View File

@@ -1,8 +0,0 @@
kind: Fixes
body: Catch all Requests Exceptions on deps install to attempt retries. Also log
the exceptions hit.
time: 2022-03-15T10:53:31.637963-05:00
custom:
Author: emmyoop
Issue: "4849"
PR: "4865"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Use cli_vars instead of context to create package and selector renderers
time: 2022-03-16T15:54:20.608384-04:00
custom:
Author: gshank
Issue: "4876"
PR: "4878"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: depend on new dbt-extractor version with fixed github links
time: 2022-03-17T13:24:18.419599-04:00
custom:
Author: nathaniel-may
Issue: "4891"
PR: "4890"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Update bumpervsion config to stop looking for missing setup.py
time: 2022-03-17T16:30:51.299919-05:00
custom:
Author: iknox-fa
Issue: "-1"
PR: "4896"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Corrected permissions settings for docker release workflow
time: 2022-03-18T09:58:46.061468-05:00
custom:
Author: iknox-fa
Issue: "4902"
PR: "4903"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: "User wasn't asked for permission to overwite a profile entry when running init inside an existing project"
time: 2022-03-21T14:05:21.238521-05:00
custom:
Author: NiallRees
Issue: "4375"
PR: "4447"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Add project name validation to `dbt init`
time: 2022-03-21T14:07:20.153071-05:00
custom:
Author: amirkdv
Issue: "4490"
PR: "4536"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Allow override of string and numeric types for adapters.
time: 2022-03-21T14:08:33.384372-05:00
custom:
Author: mdesmet
Issue: "4603"
PR: "4604"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: A change in secret environment variables won't trigger a full reparse
time: 2022-03-21T14:09:21.004543-05:00
custom:
Author: nathaniel-may
Issue: "4650"
PR: "4665"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Fix misspellings and typos in docstrings
time: 2022-03-21T14:10:07.962502-05:00
custom:
Author: kazanzhy
Issue: "4904"
PR: "4545"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Automate changelog generation with changie
time: 2022-02-18T16:13:19.882436-06:00
custom:
Author: emmyoop
Issue: "4652"
PR: "4743"

View File

@@ -1,8 +0,0 @@
kind: Under the Hood
body: Fix broken links for changelog generation and tweak GHA to only post a comment
once when changelog entry is missing.
time: 2022-03-11T10:18:51.404524-06:00
custom:
Author: emmyoop
Issue: "4848"
PR: "4857"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Add support for Python 3.10
time: 2022-03-15T15:54:36.759434-05:00
custom:
Author: iknox-fa
Issue: "4562"
PR: "4866"

View File

@@ -1,8 +0,0 @@
kind: Under the Hood
body: Enable more dialects to snapshot sources with added columns, even those that
don't support boolean datatypes
time: 2022-03-16T10:03:05.96768-05:00
custom:
Author: McKnight-42
Issue: "4488"
PR: "4871"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Testing cleanup
time: 2022-03-21T14:11:58.750419-05:00
custom:
Author: emmyoop
Issue: "3648"
PR: "4509"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Clean up test deprecation warnings
time: 2022-03-21T14:13:00.557425-05:00
custom:
Author: gshank
Issue: "3988"
PR: "4556"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Use mashumaro for serialization in event logging
time: 2022-03-21T14:13:38.758158-05:00
custom:
Author: gshank
Issue: "4504"
PR: "4505"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Drop support for Python 3.7.0 + 3.7.1
time: 2022-03-21T14:14:51.840637-05:00
custom:
Author: nathaniel-may
Issue: "4584"
PR: "4585"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Drop support for Python <3.7.2
time: 2022-03-21T14:15:23.893773-05:00
custom:
Author: jtcohen6
Issue: "4584"
PR: "4643"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Re-format codebase (except tests) using pre-commit hooks
time: 2022-03-21T14:16:10.285506-05:00
custom:
Author: iknox-fa
Issue: "3195"
PR: "4697"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Add deps module README
time: 2022-03-21T14:17:07.884314-05:00
custom:
Author: emmyoop
Issue: "4904"
PR: "4686"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Initial conversion of tests to pytest
time: 2022-03-21T14:17:47.852378-05:00
custom:
Author: gshank
Issue: "4690"
PR: "4691"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Fix errors in Windows for tests/functions
time: 2022-03-21T14:20:55.648765-05:00
custom:
Author: gshank
Issue: "4782"
PR: "4767"

View File

@@ -1,9 +0,0 @@
kind: Breaking Changes
body: 'For adapter plugin maintainers only: Internal adapter methods `set_relations_cache` + `_relations_cache_for_schemas`
each take an additional argument, for use with experimental `CACHE_SELECTED_ONLY`
config'
time: 2022-04-12T15:24:50.159572+02:00
custom:
Author: karunpoudel
Issue: "4688"
PR: "4860"

View File

@@ -1,7 +0,0 @@
kind: Docs
body: 'Fixed capitalization in UI for exposures of `type: ml`'
time: 2022-04-04T20:10:21.941618-05:00
custom:
Author: jonstacks
Issue: "4984"
PR: "4995"

View File

@@ -1,7 +0,0 @@
kind: Docs
body: List packages and tags in alphabetical order
time: 2022-04-04T20:10:57.066016-05:00
custom:
Author: pgoslatara
Issue: "4984"
PR: "4995"

View File

@@ -1,7 +0,0 @@
kind: Docs
body: Bump jekyll from 3.8.7 to 3.9.0
time: 2022-04-04T20:12:03.146811-05:00
custom:
Author: emmyoop
Issue: "4984"
PR: "4995"

View File

@@ -1,7 +0,0 @@
kind: Docs
body: Updated docker README to reflect necessity of using BuildKit
time: 2022-04-08T13:35:40.361554-05:00
custom:
Author: iknox-fa
Issue: "4990"
PR: "5018"

View File

@@ -1,8 +0,0 @@
kind: Features
body: Add `--cache_selected_only` flag to cache schema object of selected models
only.
time: 2022-03-16T00:38:47.8468296-05:00
custom:
Author: karunpoudel
Issue: "4688"
PR: "4860"

View File

@@ -1,7 +0,0 @@
kind: Features
body: Support custom names for generic tests
time: 2022-03-18T08:57:56.05584+01:00
custom:
Author: jtcohen6
Issue: "3348"
PR: "4898"

View File

@@ -1,10 +0,0 @@
kind: Features
body: 'Enable dbt jobs to run downstream models based on fresher sources. Compare
the source freshness results between previous and current state. If any source is
fresher and/or new in current vs. previous state, dbt will run and test the downstream
models in scope. Example command: `dbt build --select source_status:fresher+` '
time: 2022-03-28T13:47:43.750709-05:00
custom:
Author: sungchun12, matt-winkler, anaisvaillant
Issue: "4050"
PR: "4256"

View File

@@ -1,7 +0,0 @@
kind: Features
body: converting unique key as list tests to new pytest format
time: 2022-03-28T15:08:12.70006-05:00
custom:
Author: McKnight-42
Issue: "4882"
PR: "4958"

View File

@@ -1,9 +0,0 @@
kind: Features
body: Add a variable called selected_resources in the Jinja context containing a list
of all the resources matching the nodes for the --select, --exclude and/or --selector
parameters.
time: 2022-04-04T19:04:39.347479+02:00
custom:
Author: b-per
Issue: "3471"
PR: "5001"

View File

@@ -1,7 +0,0 @@
kind: Features
body: Support the DO_NOT_TRACK environment variable from the consoledonottrack.com initiative
time: 2022-04-06T08:03:28.877054+02:00
custom:
Author: agoblet
Issue: "3540"
PR: "5000"

View File

@@ -1,7 +0,0 @@
kind: Features
body: Add `--no-print` global flag
time: 2022-04-08T11:41:18.934373+01:00
custom:
Author: poloaraujo
Issue: "4710"
PR: "4854"

View File

@@ -1,7 +0,0 @@
kind: Features
body: add enabled as a source config
time: 2022-04-08T13:27:25.292454-04:00
custom:
Author: nathaniel-may
Issue: "3662"
PR: "5008"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Inconsistent timestamps between inserted/updated and deleted rows in snapshots
time: 2022-03-14T20:02:36.119598832+01:00
custom:
Author: kadero
Issue: "4347"
PR: "4513"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Catch more cases to retry package retrieval for deps pointing to the hub. Also start to cache the package requests.
time: 2022-03-31T14:39:23.952705-05:00
custom:
Author: emmyoop
Issue: "4849"
PR: "4982"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Make the warning message for a full event deque more descriptive
time: 2022-04-07T16:11:34.614988-05:00
custom:
Author: iknox-fa
Issue: "4962"
PR: "5011"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Fix hard delete snapshot test
time: 2022-04-08T15:55:12.552138-04:00
custom:
Author: gshank
Issue: "4916"
PR: "5020"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: add performance regression testing runner without orchestration
time: 2022-03-09T14:21:33.884043-05:00
custom:
Author: nathaniel-may
Issue: "4021"
PR: "4602"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Add Graph Compilation and Adapter Cache tracking
time: 2022-03-21T14:28:54.160087-04:00
custom:
Author: ChenyuLInx
Issue: "4625"
PR: "4912"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Create a dbt.tests.adapter release when releasing dbt and postgres
time: 2022-03-29T11:24:48.34465-04:00
custom:
Author: gshank
Issue: "4812"
PR: "4948"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: update docker image to use python 3.10.3
time: 2022-03-29T11:26:53.359066-04:00
custom:
Author: nathaniel-may
Issue: "4904"
PR: "4963"

View File

@@ -1,8 +0,0 @@
kind: Under the Hood
body: updates black to 22.3.0 which fixes dependency incompatibility when running
with precommit.
time: 2022-03-29T11:35:42.38155-04:00
custom:
Author: nathaniel-may
Issue: "4904"
PR: "4972"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Adds config util for ad-hoc creation of project objs or dicts
time: 2022-03-30T10:14:39.746196-05:00
custom:
Author: iknox-fa
Issue: "4808"
PR: "4981"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Remove TableComparison and convert existing calls to use dbt.tests.util
time: 2022-04-01T15:47:29.054759-04:00
custom:
Author: gshank
Issue: "4778"
PR: "4986"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Remove unneeded create_schema in snapshot materialization
time: 2022-04-04T14:47:08.960642+02:00
custom:
Author: jtcohen6
Issue: "4742"
PR: "4993"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Added .git-blame-ignore-revs file to mask re-formmating commits from git blame
time: 2022-04-08T13:55:25.790513-05:00
custom:
Author: iknox-fa
Issue: "5004"
PR: "5019"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Convert version tests to pytest
time: 2022-04-11T09:05:36.956877-05:00
custom:
Author: stu-k
Issue: "5024"
PR: "5026"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Updating tests and docs to show that we now support Python 3.10
time: 2022-04-11T10:37:24.806649-04:00
custom:
Author: leahwicz
Issue: "4974"
PR: "5025"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Update --version output and logic
time: 2022-04-11T15:46:26.113705-05:00
custom:
Author: stu-k
Issue: "4724"
PR: "5029"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: ""
time: 2022-04-11T21:39:14.402268-04:00
custom:
Author: leahwicz
Issue: "5033"
PR: "5032"

View File

@@ -6,20 +6,31 @@
- Do not edit this file directly. This file is auto-generated using [changie](https://github.com/miniscruff/changie). For details on how to document a change, see [the contributing guide](https://github.com/dbt-labs/dbt-core/blob/main/CONTRIBUTING.md#adding-changelog-entry)
## dbt-core 1.1.0-b1 - March 22, 2022
## dbt-core 1.1.0 - April 28, 2022
### Breaking Changes
- **Relevant to maintainers of adapter plugins _only_:** The abstractmethods `get_response` and `execute` now only return a `connection.AdapterReponse` in type hints. (Previously, they could return a string.) We encourage you to update your methods to return an object of class `AdapterResponse`, or implement a subclass specific to your adapter ([#4499](https://github.com/dbt-labs/dbt-core/issues/4499), [#4869](https://github.com/dbt-labs/dbt-core/pull/4869))
- **Relevant to maintainers of adapter plugins _only_:** The abstractmethods `get_response` and `execute` now only return a `connection.AdapterReponse` in type hints. (Previously, they could return a string.) We encourage you to update your methods to return an object of class `AdapterResponse`, or implement a subclass specific to your adapter
([#4499](https://github.com/dbt-labs/dbt-core/issues/4499), [#4869](https://github.com/dbt-labs/dbt-core/pull/4869))
- For adapter plugin maintainers only: Internal adapter methods `set_relations_cache` + `_relations_cache_for_schemas` each take an additional argument, for use with experimental `CACHE_SELECTED_ONLY` config ([#4688](https://github.com/dbt-labs/dbt-core/issues/4688), [#4860](https://github.com/dbt-labs/dbt-core/pull/4860))
### Features
- Change behaviour of `non_null` test so that it only `select`s all columns if `--store-failures` is enabled. ([#4769](https://github.com/dbt-labs/dbt-core/issues/4769), [#4777](https://github.com/dbt-labs/dbt-core/pull/4777))
- Testing framework for dbt adapter testing ([#4730](https://github.com/dbt-labs/dbt-core/issues/4730), [#4846](https://github.com/dbt-labs/dbt-core/pull/4846))
- Allow unique key to take a list implementation for postgres/redshift ([#4738](https://github.com/dbt-labs/dbt-core/issues/4738), [#4858](https://github.com/dbt-labs/dbt-core/pull/4858))
- Add `--cache_selected_only` flag to cache schema object of selected models only. ([#4688](https://github.com/dbt-labs/dbt-core/issues/4688), [#4860](https://github.com/dbt-labs/dbt-core/pull/4860))
- Support custom names for generic tests ([#3348](https://github.com/dbt-labs/dbt-core/issues/3348), [#4898](https://github.com/dbt-labs/dbt-core/pull/4898))
- Added Support for Semantic Versioning ([#4453](https://github.com/dbt-labs/dbt-core/issues/4453), [#4644](https://github.com/dbt-labs/dbt-core/pull/4644))
- New Dockerfile to support specific db adapters and platforms. See docker/README.md for details ([#4495](https://github.com/dbt-labs/dbt-core/issues/4495), [#4487](https://github.com/dbt-labs/dbt-core/pull/4487))
- Allow unique_key to take a list ([#2479](https://github.com/dbt-labs/dbt-core/issues/2479), [#4618](https://github.com/dbt-labs/dbt-core/pull/4618))
- Add `--quiet` global flag and `print` Jinja function ([#3451](https://github.com/dbt-labs/dbt-core/issues/3451), [#4701](https://github.com/dbt-labs/dbt-core/pull/4701))
- Add space before justification periods ([#4737](https://github.com/dbt-labs/dbt-core/issues/4737), [#4744](https://github.com/dbt-labs/dbt-core/pull/4744))
- Enable dbt jobs to run downstream models based on fresher sources. Compare the source freshness results between previous and current state. If any source is fresher and/or new in current vs. previous state, dbt will run and test the downstream models in scope. Example command: `dbt build --select source_status:fresher+` ([#4050](https://github.com/dbt-labs/dbt-core/issues/4050), [#4256](https://github.com/dbt-labs/dbt-core/pull/4256))
- converting unique key as list tests to new pytest format ([#4882](https://github.com/dbt-labs/dbt-core/issues/4882), [#4958](https://github.com/dbt-labs/dbt-core/pull/4958))
- Add a variable called selected_resources in the Jinja context containing a list of all the resources matching the nodes for the --select, --exclude and/or --selector parameters. ([#3471](https://github.com/dbt-labs/dbt-core/issues/3471), [#5001](https://github.com/dbt-labs/dbt-core/pull/5001))
- Support the DO_NOT_TRACK environment variable from the consoledonottrack.com initiative ([#3540](https://github.com/dbt-labs/dbt-core/issues/3540), [#5000](https://github.com/dbt-labs/dbt-core/pull/5000))
- Add `--no-print` global flag ([#4710](https://github.com/dbt-labs/dbt-core/issues/4710), [#4854](https://github.com/dbt-labs/dbt-core/pull/4854))
- add enabled as a source config ([#3662](https://github.com/dbt-labs/dbt-core/issues/3662), [#5008](https://github.com/dbt-labs/dbt-core/pull/5008))
### Fixes
- Fix bug causing empty node level meta, snapshot config errors ([#4459](https://github.com/dbt-labs/dbt-core/issues/4459), [#4726](https://github.com/dbt-labs/dbt-core/pull/4726))
- Inconsistent timestamps between inserted/updated and deleted rows in snapshots ([#4347](https://github.com/dbt-labs/dbt-core/issues/4347), [#4513](https://github.com/dbt-labs/dbt-core/pull/4513))
- Catch all Requests Exceptions on deps install to attempt retries. Also log the exceptions hit. ([#4849](https://github.com/dbt-labs/dbt-core/issues/4849), [#4865](https://github.com/dbt-labs/dbt-core/pull/4865))
- Use cli_vars instead of context to create package and selector renderers ([#4876](https://github.com/dbt-labs/dbt-core/issues/4876), [#4878](https://github.com/dbt-labs/dbt-core/pull/4878))
- depend on new dbt-extractor version with fixed github links ([#4891](https://github.com/dbt-labs/dbt-core/issues/4891), [#4890](https://github.com/dbt-labs/dbt-core/pull/4890))
@@ -30,11 +41,26 @@
- Allow override of string and numeric types for adapters. ([#4603](https://github.com/dbt-labs/dbt-core/issues/4603), [#4604](https://github.com/dbt-labs/dbt-core/pull/4604))
- A change in secret environment variables won't trigger a full reparse ([#4650](https://github.com/dbt-labs/dbt-core/issues/4650), [#4665](https://github.com/dbt-labs/dbt-core/pull/4665))
- Fix misspellings and typos in docstrings ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#4545](https://github.com/dbt-labs/dbt-core/pull/4545))
- Catch more cases to retry package retrieval for deps pointing to the hub. Also start to cache the package requests. ([#4849](https://github.com/dbt-labs/dbt-core/issues/4849), [#4982](https://github.com/dbt-labs/dbt-core/pull/4982))
- Make the warning message for a full event deque more descriptive ([#4962](https://github.com/dbt-labs/dbt-core/issues/4962), [#5011](https://github.com/dbt-labs/dbt-core/pull/5011))
- Fix hard delete snapshot test ([#4916](https://github.com/dbt-labs/dbt-core/issues/4916), [#5020](https://github.com/dbt-labs/dbt-core/pull/5020))
- Restore ability to utilize `updated_at` for check_cols snapshots ([#5076](https://github.com/dbt-labs/dbt-core/issues/5076), [#5077](https://github.com/dbt-labs/dbt-core/pull/5077))
- Use yaml renderer (with target context) for rendering selectors ([#5131](https://github.com/dbt-labs/dbt-core/issues/5131), [#5136](https://github.com/dbt-labs/dbt-core/pull/5136))
- Fix retry logic to return values after initial try ([#5023](https://github.com/dbt-labs/dbt-core/issues/5023), [#5137](https://github.com/dbt-labs/dbt-core/pull/5137))
- Scrub secret env vars from CommandError in exception stacktrace ([#5151](https://github.com/dbt-labs/dbt-core/issues/5151), [#5152](https://github.com/dbt-labs/dbt-core/pull/5152))
### Docs
- Resolve errors related to operations preventing DAG from generating in the docs. Also patch a spark issue to allow search to filter accurately past the missing columns. ([#4578](https://github.com/dbt-labs/dbt-core/issues/4578), [#4763](https://github.com/dbt-labs/dbt-core/pull/4763))
- Fixed capitalization in UI for exposures of `type: ml` ([#4984](https://github.com/dbt-labs/dbt-core/issues/4984), [#4995](https://github.com/dbt-labs/dbt-core/pull/4995))
- List packages and tags in alphabetical order ([#4984](https://github.com/dbt-labs/dbt-core/issues/4984), [#4995](https://github.com/dbt-labs/dbt-core/pull/4995))
- Bump jekyll from 3.8.7 to 3.9.0 ([#4984](https://github.com/dbt-labs/dbt-core/issues/4984), [#4995](https://github.com/dbt-labs/dbt-core/pull/4995))
- Updated docker README to reflect necessity of using BuildKit ([#4990](https://github.com/dbt-labs/dbt-core/issues/4990), [#5018](https://github.com/dbt-labs/dbt-core/pull/5018))
### Under the Hood
- Automate changelog generation with changie ([#4652](https://github.com/dbt-labs/dbt-core/issues/4652), [#4743](https://github.com/dbt-labs/dbt-core/pull/4743))
- add performance regression testing runner without orchestration ([#4021](https://github.com/dbt-labs/dbt-core/issues/4021), [#4602](https://github.com/dbt-labs/dbt-core/pull/4602))
- Fix broken links for changelog generation and tweak GHA to only post a comment once when changelog entry is missing. ([#4848](https://github.com/dbt-labs/dbt-core/issues/4848), [#4857](https://github.com/dbt-labs/dbt-core/pull/4857))
- Add support for Python 3.10 ([#4562](https://github.com/dbt-labs/dbt-core/issues/4562), [#4866](https://github.com/dbt-labs/dbt-core/pull/4866))
- Enable more dialects to snapshot sources with added columns, even those that don't support boolean datatypes ([#4488](https://github.com/dbt-labs/dbt-core/issues/4488), [#4871](https://github.com/dbt-labs/dbt-core/pull/4871))
- Add Graph Compilation and Adapter Cache tracking ([#4625](https://github.com/dbt-labs/dbt-core/issues/4625), [#4912](https://github.com/dbt-labs/dbt-core/pull/4912))
- Testing cleanup ([#3648](https://github.com/dbt-labs/dbt-core/issues/3648), [#4509](https://github.com/dbt-labs/dbt-core/pull/4509))
- Clean up test deprecation warnings ([#3988](https://github.com/dbt-labs/dbt-core/issues/3988), [#4556](https://github.com/dbt-labs/dbt-core/pull/4556))
- Use mashumaro for serialization in event logging ([#4504](https://github.com/dbt-labs/dbt-core/issues/4504), [#4505](https://github.com/dbt-labs/dbt-core/pull/4505))
@@ -44,18 +70,38 @@
- Add deps module README ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#4686](https://github.com/dbt-labs/dbt-core/pull/4686))
- Initial conversion of tests to pytest ([#4690](https://github.com/dbt-labs/dbt-core/issues/4690), [#4691](https://github.com/dbt-labs/dbt-core/pull/4691))
- Fix errors in Windows for tests/functions ([#4782](https://github.com/dbt-labs/dbt-core/issues/4782), [#4767](https://github.com/dbt-labs/dbt-core/pull/4767))
### Docs
- Resolve errors related to operations preventing DAG from generating in the docs. Also patch a spark issue to allow search to filter accurately past the missing columns. ([#4578](https://github.com/dbt-labs/dbt-core/issues/4578), [#4763](https://github.com/dbt-labs/dbt-core/pull/4763))
- Create a dbt.tests.adapter release when releasing dbt and postgres ([#4812](https://github.com/dbt-labs/dbt-core/issues/4812), [#4948](https://github.com/dbt-labs/dbt-core/pull/4948))
- update docker image to use python 3.10.3 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#4963](https://github.com/dbt-labs/dbt-core/pull/4963))
- updates black to 22.3.0 which fixes dependency incompatibility when running with precommit. ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#4972](https://github.com/dbt-labs/dbt-core/pull/4972))
- Adds config util for ad-hoc creation of project objs or dicts ([#4808](https://github.com/dbt-labs/dbt-core/issues/4808), [#4981](https://github.com/dbt-labs/dbt-core/pull/4981))
- Remove TableComparison and convert existing calls to use dbt.tests.util ([#4778](https://github.com/dbt-labs/dbt-core/issues/4778), [#4986](https://github.com/dbt-labs/dbt-core/pull/4986))
- Remove unneeded create_schema in snapshot materialization ([#4742](https://github.com/dbt-labs/dbt-core/issues/4742), [#4993](https://github.com/dbt-labs/dbt-core/pull/4993))
- Added .git-blame-ignore-revs file to mask re-formmating commits from git blame ([#5004](https://github.com/dbt-labs/dbt-core/issues/5004), [#5019](https://github.com/dbt-labs/dbt-core/pull/5019))
- Convert version tests to pytest ([#5024](https://github.com/dbt-labs/dbt-core/issues/5024), [#5026](https://github.com/dbt-labs/dbt-core/pull/5026))
- Updating tests and docs to show that we now support Python 3.10 ([#4974](https://github.com/dbt-labs/dbt-core/issues/4974), [#5025](https://github.com/dbt-labs/dbt-core/pull/5025))
- Update --version output and logic ([#4724](https://github.com/dbt-labs/dbt-core/issues/4724), [#5029](https://github.com/dbt-labs/dbt-core/pull/5029))
- ([#5033](https://github.com/dbt-labs/dbt-core/issues/5033), [#5032](https://github.com/dbt-labs/dbt-core/pull/5032))
### Contributors
- [@NiallRees](https://github.com/NiallRees) ([#4447](https://github.com/dbt-labs/dbt-core/pull/4447))
- [@alswang18](https://github.com/alswang18) ([#4644](https://github.com/dbt-labs/dbt-core/pull/4644))
- [@amirkdv](https://github.com/amirkdv) ([#4536](https://github.com/dbt-labs/dbt-core/pull/4536))
- [@ehmartens](https://github.com/ehmartens) ([#4701](https://github.com/dbt-labs/dbt-core/pull/4701))
- [@joellabes](https://github.com/joellabes) ([#4744](https://github.com/dbt-labs/dbt-core/pull/4744))
- [@kazanzhy](https://github.com/kazanzhy) ([#4545](https://github.com/dbt-labs/dbt-core/pull/4545))
- [@mdesmet](https://github.com/mdesmet) ([#4604](https://github.com/dbt-labs/dbt-core/pull/4604))
- [@versusfacit](https://github.com/versusfacit) ([#4869](https://github.com/dbt-labs/dbt-core/pull/4869))
- [@willbowditch](https://github.com/willbowditch) ([#4777](https://github.com/dbt-labs/dbt-core/pull/4777))
- [@NiallRees](https://github.com/NiallRees) ([#4447](https://github.com/dbt-labs/dbt-core/pull/4447))
- [@agoblet](https://github.com/agoblet) ([#5000](https://github.com/dbt-labs/dbt-core/pull/5000))
- [@alswang18](https://github.com/alswang18) ([#4644](https://github.com/dbt-labs/dbt-core/pull/4644))
- [@amirkdv](https://github.com/amirkdv) ([#4536](https://github.com/dbt-labs/dbt-core/pull/4536))
- [@anaisvaillant](https://github.com/anaisvaillant) ([#4256](https://github.com/dbt-labs/dbt-core/pull/4256))
- [@b-per](https://github.com/b-per) ([#5001](https://github.com/dbt-labs/dbt-core/pull/5001))
- [@dbeatty10](https://github.com/dbeatty10) ([#5077](https://github.com/dbt-labs/dbt-core/pull/5077))
- [@ehmartens](https://github.com/ehmartens) ([#4701](https://github.com/dbt-labs/dbt-core/pull/4701))
- [@joellabes](https://github.com/joellabes) ([#4744](https://github.com/dbt-labs/dbt-core/pull/4744))
- [@jonstacks](https://github.com/jonstacks) ([#4995](https://github.com/dbt-labs/dbt-core/pull/4995))
- [@kadero](https://github.com/kadero) ([#4513](https://github.com/dbt-labs/dbt-core/pull/4513))
- [@karunpoudel](https://github.com/karunpoudel) ([#4860](https://github.com/dbt-labs/dbt-core/pull/4860), [#4860](https://github.com/dbt-labs/dbt-core/pull/4860))
- [@kazanzhy](https://github.com/kazanzhy) ([#4545](https://github.com/dbt-labs/dbt-core/pull/4545))
- [@matt-winkler](https://github.com/matt-winkler) ([#4256](https://github.com/dbt-labs/dbt-core/pull/4256))
- [@mdesmet](https://github.com/mdesmet) ([#4604](https://github.com/dbt-labs/dbt-core/pull/4604))
- [@pgoslatara](https://github.com/pgoslatara) ([#4995](https://github.com/dbt-labs/dbt-core/pull/4995))
- [@poloaraujo](https://github.com/poloaraujo) ([#4854](https://github.com/dbt-labs/dbt-core/pull/4854))
- [@sungchun12](https://github.com/sungchun12) ([#4256](https://github.com/dbt-labs/dbt-core/pull/4256))
- [@willbowditch](https://github.com/willbowditch) ([#4777](https://github.com/dbt-labs/dbt-core/pull/4777))
## Previous Releases

View File

@@ -28,7 +28,7 @@ def _is_commit(revision: str) -> bool:
def _raise_git_cloning_error(repo, revision, error):
stderr = error.stderr.decode("utf-8").strip()
stderr = error.stderr.strip()
if "usage: git" in stderr:
stderr = stderr.split("\nusage: git")[0]
if re.match("fatal: destination path '(.+)' already exists", stderr):
@@ -115,8 +115,8 @@ def checkout(cwd, repo, revision=None):
try:
return _checkout(cwd, repo, revision)
except CommandResultError as exc:
stderr = exc.stderr.decode("utf-8").strip()
bad_package_spec(repo, revision, stderr)
stderr = exc.stderr.strip()
bad_package_spec(repo, revision, stderr)
def get_current_sha(cwd):
@@ -142,7 +142,7 @@ def clone_and_checkout(
subdirectory=subdirectory,
)
except CommandResultError as exc:
err = exc.stderr.decode("utf-8")
err = exc.stderr
exists = re.match("fatal: destination path '(.+)' already exists", err)
if not exists:
raise_git_cloning_problem(repo)

View File

@@ -114,12 +114,10 @@ class DbtProjectYamlRenderer(BaseRenderer):
def name(self):
"Project config"
# Uses SecretRenderer
def get_package_renderer(self) -> BaseRenderer:
return PackageRenderer(self.ctx_obj.cli_vars)
def get_selector_renderer(self) -> BaseRenderer:
return SelectorRenderer(self.ctx_obj.cli_vars)
def render_project(
self,
project: Dict[str, Any],
@@ -136,8 +134,7 @@ class DbtProjectYamlRenderer(BaseRenderer):
return package_renderer.render_data(packages)
def render_selectors(self, selectors: Dict[str, Any]):
selector_renderer = self.get_selector_renderer()
return selector_renderer.render_data(selectors)
return self.render_data(selectors)
def render_entry(self, value: Any, keypath: Keypath) -> Any:
result = super().render_entry(value, keypath)
@@ -165,18 +162,10 @@ class DbtProjectYamlRenderer(BaseRenderer):
return True
class SelectorRenderer(BaseRenderer):
@property
def name(self):
return "Selector config"
class SecretRenderer(BaseRenderer):
def __init__(self, cli_vars: Optional[Dict[str, Any]] = None) -> None:
def __init__(self, cli_vars: Dict[str, Any] = {}) -> None:
# Generate contexts here because we want to save the context
# object in order to retrieve the env_vars.
if cli_vars is None:
cli_vars = {}
self.ctx_obj = SecretContext(cli_vars)
context = self.ctx_obj.to_dict()
super().__init__(context)

View File

@@ -3,7 +3,7 @@ from typing import Dict, Any, Union
from dbt.clients.yaml_helper import yaml, Loader, Dumper, load_yaml_text # noqa: F401
from dbt.dataclass_schema import ValidationError
from .renderer import SelectorRenderer
from .renderer import BaseRenderer
from dbt.clients.system import (
load_file_contents,
@@ -57,7 +57,7 @@ class SelectorConfig(Dict[str, Dict[str, Union[SelectionSpec, bool]]]):
def render_from_dict(
cls,
data: Dict[str, Any],
renderer: SelectorRenderer,
renderer: BaseRenderer,
) -> "SelectorConfig":
try:
rendered = renderer.render_data(data)
@@ -72,7 +72,7 @@ class SelectorConfig(Dict[str, Dict[str, Union[SelectionSpec, bool]]]):
def from_path(
cls,
path: Path,
renderer: SelectorRenderer,
renderer: BaseRenderer,
) -> "SelectorConfig":
try:
data = load_yaml_text(load_file_contents(str(path)))

View File

@@ -62,6 +62,8 @@ def get_project_config(
user_config = read_user_config(flags.PROFILES_DIR)
# Update flags
flags.set_from_args(args, user_config)
if cli_vars is None:
cli_vars = {}
profile = Profile.render_from_args(args, ProfileRenderer(cli_vars), profile_name)
# Generate a project
project = Project.from_project_root(

View File

@@ -383,10 +383,11 @@ class FailedToConnectException(DatabaseException):
class CommandError(RuntimeException):
def __init__(self, cwd, cmd, message="Error running command"):
cmd_scrubbed = list(scrub_secrets(cmd_txt, env_secrets()) for cmd_txt in cmd)
super().__init__(message)
self.cwd = cwd
self.cmd = cmd
self.args = (cwd, cmd, message)
self.cmd = cmd_scrubbed
self.args = (cwd, cmd_scrubbed, message)
def __str__(self):
if len(self.cmd) == 0:
@@ -411,9 +412,9 @@ class CommandResultError(CommandError):
def __init__(self, cwd, cmd, returncode, stdout, stderr, message="Got a non-zero returncode"):
super().__init__(cwd, cmd, message)
self.returncode = returncode
self.stdout = stdout
self.stderr = stderr
self.args = (cwd, cmd, returncode, stdout, stderr, message)
self.stdout = scrub_secrets(stdout.decode("utf-8"), env_secrets())
self.stderr = scrub_secrets(stderr.decode("utf-8"), env_secrets())
self.args = (cwd, self.cmd, returncode, self.stdout, self.stderr, message)
def __str__(self):
return "{} running: {}".format(self.msg, self.cmd)
@@ -704,7 +705,6 @@ def missing_materialization(model, adapter_type):
def bad_package_spec(repo, spec, error_message):
msg = "Error checking out spec='{}' for repo {}\n{}".format(spec, repo, error_message)
raise InternalException(scrub_secrets(msg, env_secrets()))

View File

@@ -132,7 +132,7 @@
{% set check_cols_config = config['check_cols'] %}
{% set primary_key = config['unique_key'] %}
{% set invalidate_hard_deletes = config.get('invalidate_hard_deletes', false) %}
{% set updated_at = snapshot_get_time() %}
{% set updated_at = config.get('updated_at', snapshot_get_time()) %}
{% set column_added = false %}

View File

@@ -107,6 +107,9 @@ def test_data_dir(request):
# This contains the profile target information, for simplicity in setting
# up different profiles, particularly in the adapter repos.
# Note: because we load the profile to create the adapter, this
# fixture can't be used to test vars and env_vars or errors. The
# profile must be written out after the test starts.
@pytest.fixture(scope="class")
def dbt_profile_target():
return {

View File

@@ -23,6 +23,7 @@ from dbt.events.test_types import IntegrationTestDebug
# read_file
# get_artifact
# update_config_file
# write_config_file
# get_unique_ids_in_results
# check_result_nodes_by_name
# check_result_nodes_by_unique_id
@@ -143,6 +144,13 @@ def update_config_file(updates, *paths):
write_file(new_yaml, *paths)
# Write new config file
def write_config_file(data, *paths):
if type(data) is dict:
data = yaml.safe_dump(data)
write_file(data, *paths)
# Get the unique_ids in dbt command results
def get_unique_ids_in_results(results):
unique_ids = []
@@ -273,13 +281,15 @@ def check_relation_types(adapter, relation_to_type):
# by doing a separate call for each set of tables/relations.
# Wraps check_relations_equal_with_relations by creating relations
# from the list of names passed in.
def check_relations_equal(adapter, relation_names):
def check_relations_equal(adapter, relation_names, compare_snapshot_cols=False):
if len(relation_names) < 2:
raise TestProcessingException(
"Not enough relations to compare",
)
relations = [relation_from_name(adapter, name) for name in relation_names]
return check_relations_equal_with_relations(adapter, relations)
return check_relations_equal_with_relations(
adapter, relations, compare_snapshot_cols=compare_snapshot_cols
)
# This can be used when checking relations in different schemas, by supplying
@@ -288,16 +298,17 @@ def check_relations_equal(adapter, relation_names):
# adapter.get_columns_in_relation
# adapter.get_rows_different_sql
# adapter.execute
def check_relations_equal_with_relations(adapter, relations):
def check_relations_equal_with_relations(adapter, relations, compare_snapshot_cols=False):
with get_connection(adapter):
basis, compares = relations[0], relations[1:]
# Skip columns starting with "dbt_" because we don't want to
# compare those, since they are time sensitive
# (unless comparing "dbt_" snapshot columns is explicitly enabled)
column_names = [
c.name
for c in adapter.get_columns_in_relation(basis)
if not c.name.lower().startswith("dbt_")
if not c.name.lower().startswith("dbt_") or compare_snapshot_cols
]
for relation in compares:

View File

@@ -615,7 +615,7 @@ def _connection_exception_retry(fn, max_attempts: int, attempt: int = 0):
fire_event(RecordRetryException(exc=exc))
fire_event(RetryExternalCall(attempt=attempt, max=max_attempts))
time.sleep(1)
_connection_exception_retry(fn, max_attempts, attempt + 1)
return _connection_exception_retry(fn, max_attempts, attempt + 1)
else:
raise ConnectionException("External connection exception occurred: " + str(exc))

View File

@@ -234,5 +234,5 @@ def _get_adapter_plugin_names() -> Iterator[str]:
yield plugin_name
__version__ = "1.1.0b1"
__version__ = "1.1.0"
installed = get_installed_version()

View File

@@ -273,12 +273,12 @@ def parse_args(argv=None):
parser.add_argument("adapter")
parser.add_argument("--title-case", "-t", default=None)
parser.add_argument("--dependency", action="append")
parser.add_argument("--dbt-core-version", default="1.1.0b1")
parser.add_argument("--dbt-core-version", default="1.1.0")
parser.add_argument("--email")
parser.add_argument("--author")
parser.add_argument("--url")
parser.add_argument("--sql", action="store_true")
parser.add_argument("--package-version", default="1.1.0b1")
parser.add_argument("--package-version", default="1.1.0")
parser.add_argument("--project-version", default="1.0")
parser.add_argument("--no-dependency", action="store_false", dest="set_dependency")
parsed = parser.parse_args()

View File

@@ -25,7 +25,7 @@ with open(os.path.join(this_directory, "README.md")) as f:
package_name = "dbt-core"
package_version = "1.1.0b1"
package_version = "1.1.0"
description = """With dbt, data analysts and engineers can build analytics \
the way engineers build applications."""

View File

@@ -14,12 +14,12 @@ FROM --platform=$build_for python:3.10.3-slim-bullseye as base
# N.B. The refs updated automagically every release via bumpversion
# N.B. dbt-postgres is currently found in the core codebase so a value of dbt-core@<some_version> is correct
ARG dbt_core_ref=dbt-core@v1.1.0b1
ARG dbt_postgres_ref=dbt-core@v1.1.0b1
ARG dbt_redshift_ref=dbt-redshift@v1.0.0
ARG dbt_bigquery_ref=dbt-bigquery@v1.0.0
ARG dbt_snowflake_ref=dbt-snowflake@v1.0.0
ARG dbt_spark_ref=dbt-spark@v1.0.0
ARG dbt_core_ref=dbt-core@v1.1.0
ARG dbt_postgres_ref=dbt-core@v1.1.0
ARG dbt_redshift_ref=dbt-redshift@v1.1.0
ARG dbt_bigquery_ref=dbt-bigquery@v1.1.0
ARG dbt_snowflake_ref=dbt-snowflake@v1.1.0
ARG dbt_spark_ref=dbt-spark@v1.1.0
# special case args
ARG dbt_spark_version=all
ARG dbt_third_party

View File

@@ -1 +1 @@
version = "1.1.0b1"
version = "1.1.0"

View File

@@ -41,7 +41,7 @@ def _dbt_psycopg2_name():
package_name = "dbt-postgres"
package_version = "1.1.0b1"
package_version = "1.1.0"
description = """The postgres adpter plugin for dbt (data build tool)"""
this_directory = os.path.abspath(os.path.dirname(__file__))

View File

@@ -221,6 +221,36 @@ class TestAllowSecretProfilePackage(DBTIntegrationTest):
self.assertFalse("first_dependency" in log_output)
class TestCloneFailSecretScrubbed(DBTIntegrationTest):
def setUp(self):
os.environ[SECRET_ENV_PREFIX + "GIT_TOKEN"] = "abc123"
DBTIntegrationTest.setUp(self)
@property
def packages_config(self):
return {
"packages": [
{"git": "https://fakeuser:{{ env_var('DBT_ENV_SECRET_GIT_TOKEN') }}@github.com/dbt-labs/fake-repo.git"},
]
}
@property
def schema(self):
return "context_vars_013"
@property
def models(self):
return "models"
@use_profile('postgres')
def test_postgres_fail_clone_with_scrubbing(self):
with self.assertRaises(dbt.exceptions.InternalException) as exc:
_, log_output = self.run_dbt_and_capture(['deps'])
assert "abc123" not in str(exc.exception)
class TestEmitWarning(DBTIntegrationTest):
@property
def schema(self):

View File

@@ -1,6 +0,0 @@
select
'{{ var("variable_1") }}'::varchar as var_1,
'{{ var("variable_2")[0] }}'::varchar as var_2,
'{{ var("variable_3")["value"] }}'::varchar as var_3

View File

@@ -1,19 +0,0 @@
version: 2
models:
- name: complex_model
columns:
- name: var_1
tests:
- accepted_values:
values:
- abc
- name: var_2
tests:
- accepted_values:
values:
- def
- name: var_3
tests:
- accepted_values:
values:
- jkl

View File

@@ -1,9 +0,0 @@
version: 2
models:
- name: test_vars
columns:
- name: field
tests:
- accepted_values:
values:
- override

View File

@@ -1,3 +0,0 @@
select '{{ var("required") }}'::varchar as field

View File

@@ -1,9 +0,0 @@
version: 2
models:
- name: simple_model
columns:
- name: simple
tests:
- accepted_values:
values:
- abc

View File

@@ -1,4 +0,0 @@
select
'{{ var("simple") }}'::varchar as simple

View File

@@ -1,57 +0,0 @@
from test.integration.base import DBTIntegrationTest, use_profile
class TestCLIVarOverride(DBTIntegrationTest):
@property
def schema(self):
return "cli_vars_028"
@property
def models(self):
return "models_override"
@property
def project_config(self):
return {
'config-version': 2,
'vars': {
'required': 'present',
},
}
@use_profile('postgres')
def test__postgres_overriden_vars_global(self):
self.use_default_project()
self.use_profile('postgres')
# This should be "override"
self.run_dbt(["run", "--vars", "{required: override}"])
self.run_dbt(["test"])
class TestCLIVarOverridePorject(DBTIntegrationTest):
@property
def schema(self):
return "cli_vars_028"
@property
def models(self):
return "models_override"
@property
def project_config(self):
return {
'config-version': 2,
'vars': {
'test': {
'required': 'present',
},
},
}
@use_profile('postgres')
def test__postgres_overriden_vars_project_level(self):
# This should be "override"
self.run_dbt(["run", "--vars", "{required: override}"])
self.run_dbt(["test"])

View File

@@ -1,68 +0,0 @@
from test.integration.base import DBTIntegrationTest, use_profile
import yaml
import json
class TestCLIVars(DBTIntegrationTest):
@property
def schema(self):
return "cli_vars_028"
@property
def models(self):
return "models_complex"
@use_profile('postgres')
def test__postgres_cli_vars_longform(self):
self.use_profile('postgres')
self.use_default_project()
cli_vars = {
"variable_1": "abc",
"variable_2": ["def", "ghi"],
"variable_3": {
"value": "jkl"
}
}
results = self.run_dbt(["run", "--vars", yaml.dump(cli_vars)])
self.assertEqual(len(results), 1)
results = self.run_dbt(["test", "--vars", yaml.dump(cli_vars)])
self.assertEqual(len(results), 3)
class TestCLIVarsSimple(DBTIntegrationTest):
@property
def schema(self):
return "cli_vars_028"
@property
def models(self):
return "models_simple"
@use_profile('postgres')
def test__postgres_cli_vars_shorthand(self):
self.use_profile('postgres')
self.use_default_project()
results = self.run_dbt(["run", "--vars", "simple: abc"])
self.assertEqual(len(results), 1)
results = self.run_dbt(["test", "--vars", "simple: abc"])
self.assertEqual(len(results), 1)
@use_profile('postgres')
def test__postgres_cli_vars_longer(self):
self.use_profile('postgres')
self.use_default_project()
results = self.run_dbt(["run", "--vars", "{simple: abc, unused: def}"])
self.assertEqual(len(results), 1)
results = self.run_dbt(["test", "--vars", "{simple: abc, unused: def}"])
self.assertEqual(len(results), 1)
run_results = _read_json('./target/run_results.json')
self.assertEqual(run_results['args']['vars'], "{simple: abc, unused: def}")
def _read_json(path):
# read json generated by dbt.
with open(path) as fp:
return json.load(fp)

View File

@@ -1 +1 @@
version = "1.1.0b1"
version = "1.1.0"

View File

@@ -20,7 +20,7 @@ except ImportError:
package_name = "dbt-tests-adapter"
package_version = "1.1.0b1"
package_version = "1.1.0"
description = """The dbt adapter tests for adapter plugins"""
this_directory = os.path.abspath(os.path.dirname(__file__))

View File

@@ -0,0 +1,68 @@
import pytest
from dbt.tests.util import run_dbt
models_override__schema_yml = """
version: 2
models:
- name: test_vars
columns:
- name: field
tests:
- accepted_values:
values:
- override
"""
models_override__test_vars_sql = """
select '{{ var("required") }}'::varchar as field
"""
# Tests that cli vars override vars set in the project config
class TestCLIVarOverride:
@pytest.fixture(scope="class")
def models(self):
return {
"schema.yml": models_override__schema_yml,
"test_vars.sql": models_override__test_vars_sql,
}
@pytest.fixture(scope="class")
def project_config_update(self):
return {
"vars": {
"required": "present",
},
}
def test__override_vars_global(self, project):
run_dbt(["run", "--vars", "{required: override}"])
run_dbt(["test"])
# This one switches to setting a var in 'test'
class TestCLIVarOverridePorject:
@pytest.fixture(scope="class")
def models(self):
return {
"schema.yml": models_override__schema_yml,
"test_vars.sql": models_override__test_vars_sql,
}
@pytest.fixture(scope="class")
def project_config_update(self):
return {
"vars": {
"test": {
"required": "present",
},
},
}
def test__override_vars_project_level(self, project):
# This should be "override"
run_dbt(["run", "--vars", "{required: override}"])
run_dbt(["test"])

View File

@@ -0,0 +1,212 @@
import pytest
import yaml
from dbt.tests.util import run_dbt, get_artifact, write_config_file
from dbt.exceptions import RuntimeException, CompilationException
models_complex__schema_yml = """
version: 2
models:
- name: complex_model
columns:
- name: var_1
tests:
- accepted_values:
values:
- abc
- name: var_2
tests:
- accepted_values:
values:
- def
- name: var_3
tests:
- accepted_values:
values:
- jkl
"""
models_complex__complex_model_sql = """
select
'{{ var("variable_1") }}'::varchar as var_1,
'{{ var("variable_2")[0] }}'::varchar as var_2,
'{{ var("variable_3")["value"] }}'::varchar as var_3
"""
models_simple__schema_yml = """
version: 2
models:
- name: simple_model
columns:
- name: simple
tests:
- accepted_values:
values:
- abc
"""
models_simple__simple_model_sql = """
select
'{{ var("simple") }}'::varchar as simple
"""
really_simple_model_sql = """
select 'abc' as simple
"""
class TestCLIVars:
@pytest.fixture(scope="class")
def models(self):
return {
"schema.yml": models_complex__schema_yml,
"complex_model.sql": models_complex__complex_model_sql,
}
def test__cli_vars_longform(self, project):
cli_vars = {
"variable_1": "abc",
"variable_2": ["def", "ghi"],
"variable_3": {"value": "jkl"},
}
results = run_dbt(["run", "--vars", yaml.dump(cli_vars)])
assert len(results) == 1
results = run_dbt(["test", "--vars", yaml.dump(cli_vars)])
assert len(results) == 3
class TestCLIVarsSimple:
@pytest.fixture(scope="class")
def models(self):
return {
"schema.yml": models_simple__schema_yml,
"simple_model.sql": models_simple__simple_model_sql,
}
def test__cli_vars_shorthand(self, project):
results = run_dbt(["run", "--vars", "simple: abc"])
assert len(results) == 1
results = run_dbt(["test", "--vars", "simple: abc"])
assert len(results) == 1
def test__cli_vars_longer(self, project):
results = run_dbt(["run", "--vars", "{simple: abc, unused: def}"])
assert len(results) == 1
results = run_dbt(["test", "--vars", "{simple: abc, unused: def}"])
assert len(results) == 1
run_results = get_artifact(project.project_root, "target", "run_results.json")
assert run_results["args"]["vars"] == "{simple: abc, unused: def}"
class TestCLIVarsProfile:
@pytest.fixture(scope="class")
def models(self):
return {
"schema.yml": models_simple__schema_yml,
"simple_model.sql": really_simple_model_sql,
}
def test_cli_vars_in_profile(self, project, dbt_profile_data):
profile = dbt_profile_data
profile["test"]["outputs"]["default"]["host"] = "{{ var('db_host') }}"
write_config_file(profile, project.profiles_dir, "profiles.yml")
with pytest.raises(RuntimeException):
results = run_dbt(["run"])
results = run_dbt(["run", "--vars", "db_host: localhost"])
assert len(results) == 1
class TestCLIVarsPackages:
@pytest.fixture(scope="class")
def models(self):
return {
"schema.yml": models_simple__schema_yml,
"simple_model.sql": really_simple_model_sql,
}
@pytest.fixture(scope="class")
def packages_config(self):
return {
"packages": [
{
"git": "https://github.com/dbt-labs/dbt-integration-project",
"revision": "1.1",
}
]
}
def test_cli_vars_in_packages(self, project, packages_config):
# Run working deps and run commands
run_dbt(["deps"])
results = run_dbt(["run"])
assert len(results) == 1
# Change packages.yml to contain a var
packages = packages_config
packages["packages"][0]["revision"] = "{{ var('dip_version') }}"
write_config_file(packages, project.project_root, "packages.yml")
# Without vars args deps fails
with pytest.raises(RuntimeException):
run_dbt(["deps"])
# With vars arg deps succeeds
results = run_dbt(["deps", "--vars", "dip_version: 1.1"])
assert results is None
# Do a dbt run command so adapter exists
results = run_dbt(["run", "--vars", "dip_version: 1.1"], expect_pass=False)
assert len(results) == 4
initial_selectors_yml = """
selectors:
- name: dev_defer_snapshots
default: "{{ target.name == 'dev' | as_bool }}"
definition:
method: fqn
value: '*'
exclude:
- method: config.materialized
value: snapshot
"""
var_selectors_yml = """
selectors:
- name: dev_defer_snapshots
default: "{{ var('snapshot_target') == 'dev' | as_bool }}"
definition:
method: fqn
value: '*'
exclude:
- method: config.materialized
value: snapshot
"""
class TestCLIVarsSelectors:
@pytest.fixture(scope="class")
def models(self):
return {
"schema.yml": models_simple__schema_yml,
"simple_model.sql": really_simple_model_sql,
}
@pytest.fixture(scope="class")
def selectors(self):
return initial_selectors_yml
def test_vars_in_selectors(self, project):
# initially runs ok
results = run_dbt(["run"])
assert len(results) == 1
# Update the selectors.yml file to have a var
write_config_file(var_selectors_yml, project.project_root, "selectors.yml")
with pytest.raises(CompilationException):
run_dbt(["run"])
# Var in cli_vars works
results = run_dbt(["run", "--vars", "snapshot_target: dev"])
assert len(results) == 1

View File

@@ -0,0 +1,113 @@
import pytest
from dbt.tests.util import run_dbt, check_relations_equal
snapshot_sql = """
{% snapshot snapshot_check_cols_updated_at_actual %}
{{
config(
target_database=database,
target_schema=schema,
unique_key='id',
strategy='check',
check_cols='all',
updated_at="'" ~ var("updated_at") ~ "'::timestamp",
)
}}
{% if var('version') == 1 %}
select 'a' as id, 10 as counter, '2016-01-01T00:00:00Z'::timestamp as timestamp_col union all
select 'b' as id, 20 as counter, '2016-01-01T00:00:00Z'::timestamp as timestamp_col
{% elif var('version') == 2 %}
select 'a' as id, 30 as counter, '2016-01-02T00:00:00Z'::timestamp as timestamp_col union all
select 'b' as id, 20 as counter, '2016-01-01T00:00:00Z'::timestamp as timestamp_col union all
select 'c' as id, 40 as counter, '2016-01-02T00:00:00Z'::timestamp as timestamp_col
{% else %}
select 'a' as id, 30 as counter, '2016-01-02T00:00:00Z'::timestamp as timestamp_col union all
select 'c' as id, 40 as counter, '2016-01-02T00:00:00Z'::timestamp as timestamp_col
{% endif %}
{% endsnapshot %}
"""
expected_csv = """
id,counter,timestamp_col,dbt_scd_id,dbt_updated_at,dbt_valid_from,dbt_valid_to
a,10,2016-01-01 00:00:00.000,927354aa091feffd9437ead0bdae7ae1,2016-07-01 00:00:00.000,2016-07-01 00:00:00.000,2016-07-02 00:00:00.000
b,20,2016-01-01 00:00:00.000,40ace4cbf8629f1720ec8a529ed76f8c,2016-07-01 00:00:00.000,2016-07-01 00:00:00.000,
a,30,2016-01-02 00:00:00.000,e9133f2b302c50e36f43e770944cec9b,2016-07-02 00:00:00.000,2016-07-02 00:00:00.000,
c,40,2016-01-02 00:00:00.000,09d33d35101e788c152f65d0530b6837,2016-07-02 00:00:00.000,2016-07-02 00:00:00.000,
""".lstrip()
@pytest.fixture(scope="class")
def snapshots():
return {"snapshot_check_cols_updated_at_actual.sql": snapshot_sql}
@pytest.fixture(scope="class")
def seeds():
return {"snapshot_check_cols_updated_at_expected.csv": expected_csv}
@pytest.fixture(scope="class")
def project_config_update():
return {
"seeds": {
"quote_columns": False,
"test": {
"snapshot_check_cols_updated_at_expected": {
"+column_types": {
"timestamp_col": "timestamp without time zone",
"dbt_updated_at": "timestamp without time zone",
"dbt_valid_from": "timestamp without time zone",
"dbt_valid_to": "timestamp without time zone",
},
},
},
},
}
def test_simple_snapshot(project):
"""
Test that the `dbt_updated_at` column reflects the `updated_at` timestamp expression in the config.
Approach:
1. Create a table that represents the expected data after a series of snapshots
- Use dbt seed to create the expected relation (`snapshot_check_cols_updated_at_expected`)
2. Execute a series of snapshots to create the data
- Use a series of (3) dbt snapshot commands to create the actual relation (`snapshot_check_cols_updated_at_actual`)
- The logic can switch between 3 different versions of the data (depending on the `version` number)
- The `updated_at` value is passed in via `--vars` and cast to a timestamp in the snapshot config
3. Compare the two relations for equality
"""
# 1. Create a table that represents the expected data after a series of snapshots
results = run_dbt(["seed", "--show", "--vars", "{version: 1, updated_at: 2016-07-01}"])
assert len(results) == 1
# 2. Execute a series of snapshots to create the data
# Snapshot day 1
results = run_dbt(["snapshot", "--vars", "{version: 1, updated_at: 2016-07-01}"])
assert len(results) == 1
# Snapshot day 2
results = run_dbt(["snapshot", "--vars", "{version: 2, updated_at: 2016-07-02}"])
assert len(results) == 1
# Snapshot day 3
results = run_dbt(["snapshot", "--vars", "{version: 3, updated_at: 2016-07-03}"])
assert len(results) == 1
# 3. Compare the two relations for equality
check_relations_equal(
project.adapter,
["snapshot_check_cols_updated_at_actual", "snapshot_check_cols_updated_at_expected"],
compare_snapshot_cols=True,
)

View File

@@ -0,0 +1,59 @@
import functools
import pytest
from requests.exceptions import RequestException
from dbt.exceptions import ConnectionException
from dbt.utils import _connection_exception_retry
def no_retry_fn():
return "success"
class TestNoRetries:
def test_no_retry(self):
fn_to_retry = functools.partial(no_retry_fn)
result = _connection_exception_retry(fn_to_retry, 3)
expected = "success"
assert result == expected
def no_success_fn():
raise RequestException("You'll never pass")
return "failure"
class TestMaxRetries:
def test_no_retry(self):
fn_to_retry = functools.partial(no_success_fn)
with pytest.raises(ConnectionException):
_connection_exception_retry(fn_to_retry, 3)
def single_retry_fn():
global counter
if counter == 0:
counter += 1
raise RequestException("You won't pass this one time")
elif counter == 1:
counter += 1
return "success on 2"
return "How did we get here?"
class TestSingleRetry:
def test_no_retry(self):
global counter
counter = 0
fn_to_retry = functools.partial(single_retry_fn)
result = _connection_exception_retry(fn_to_retry, 3)
expected = "success on 2"
# We need to test the return value here, not just that it did not throw an error.
# If the value is not being passed it causes cryptic errors
assert result == expected
assert counter == 2