mirror of
https://github.com/dbt-labs/dbt-core
synced 2025-12-17 19:31:34 +00:00
Compare commits
5 Commits
adding-sem
...
v1.2.0
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c7652f6a33 | ||
|
|
a2a531722e | ||
|
|
7656ffb9e6 | ||
|
|
3f3792882c | ||
|
|
0db634d12f |
@@ -1,5 +1,5 @@
|
||||
[bumpversion]
|
||||
current_version = 1.2.0b1
|
||||
current_version = 1.2.0
|
||||
parse = (?P<major>\d+)
|
||||
\.(?P<minor>\d+)
|
||||
\.(?P<patch>\d+)
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
## dbt-core 1.2.0-b1 - June 24, 2022
|
||||
## dbt-core 1.2.0 - July 26, 2022
|
||||
### Features
|
||||
- Add selector method when reading selector definitions ([#4821](https://github.com/dbt-labs/dbt-core/issues/4821), [#4827](https://github.com/dbt-labs/dbt-core/pull/4827))
|
||||
- Add set and zip function to contexts ([#2345](https://github.com/dbt-labs/dbt-core/issues/2345), [#5107](https://github.com/dbt-labs/dbt-core/pull/5107))
|
||||
@@ -10,6 +10,11 @@
|
||||
- Move cross-db macros from dbt-utils into dbt-core global project ([#4813](https://github.com/dbt-labs/dbt-core/issues/4813), [#5265](https://github.com/dbt-labs/dbt-core/pull/5265))
|
||||
- Prettify duration message at the end of execution ([#5253](https://github.com/dbt-labs/dbt-core/issues/5253), [#5364](https://github.com/dbt-labs/dbt-core/pull/5364))
|
||||
- Early return from dbt init if no available adapters ([#5365](https://github.com/dbt-labs/dbt-core/issues/5365), [#5366](https://github.com/dbt-labs/dbt-core/pull/5366))
|
||||
- Allow customizing `target-path` and `log-path` through environment variables and CLI flags. ([#5399](https://github.com/dbt-labs/dbt-core/issues/5399), [#5402](https://github.com/dbt-labs/dbt-core/pull/5402))
|
||||
- Move type_* macros from dbt-utils into dbt-core, with tests ([#5317](https://github.com/dbt-labs/dbt-core/issues/5317), [#5428](https://github.com/dbt-labs/dbt-core/pull/5428))
|
||||
- Add support for ratio metrics ([#4884](https://github.com/dbt-labs/dbt-core/issues/4884), [#5027](https://github.com/dbt-labs/dbt-core/pull/5027))
|
||||
- Allow users to define grants as a reasonable default in the dbt_project.yml or within each model sql or yml file combined. ([#5263](https://github.com/dbt-labs/dbt-core/issues/5263), [#5369](https://github.com/dbt-labs/dbt-core/pull/5369))
|
||||
- Add reusable function for retrying adapter connections. Utilize said function to add retries for Postgres (and Redshift). ([#5022](https://github.com/dbt-labs/dbt-core/issues/5022), [#5432](https://github.com/dbt-labs/dbt-core/pull/5432))
|
||||
### Fixes
|
||||
- Adding new cols to check_cols in snapshots ([#3146](https://github.com/dbt-labs/dbt-core/issues/3146), [#4893](https://github.com/dbt-labs/dbt-core/pull/4893))
|
||||
- Truncate relation names when appending a suffix that will result in len > 63 characters using make_temp_relation and make_backup_relation macros ([#2869](https://github.com/dbt-labs/dbt-core/issues/2869), [#4921](https://github.com/dbt-labs/dbt-core/pull/4921))
|
||||
@@ -29,7 +34,19 @@
|
||||
- Change node ancestor/descendant algo, fixes issue where downstream models aren't run when using networkx >= 2.8.1 ([#5286](https://github.com/dbt-labs/dbt-core/issues/5286), [#5326](https://github.com/dbt-labs/dbt-core/pull/5326))
|
||||
- Fixing Windows color regression ([#5191](https://github.com/dbt-labs/dbt-core/issues/5191), [#5327](https://github.com/dbt-labs/dbt-core/pull/5327))
|
||||
- Define compatibility for older manifest versions when using state: selection methods ([#5213](https://github.com/dbt-labs/dbt-core/issues/5213), [#5346](https://github.com/dbt-labs/dbt-core/pull/5346))
|
||||
- Add inheritance to materialization macro resolution ([#4646](https://github.com/dbt-labs/dbt-core/issues/4646), [#5348](https://github.com/dbt-labs/dbt-core/pull/5348))
|
||||
- Improve pluralizations for Documentation and SqlOperation NodeTypes ([#5352](https://github.com/dbt-labs/dbt-core/issues/5352), [#5356](https://github.com/dbt-labs/dbt-core/pull/5356))
|
||||
- Properly use quotes for Snowflake snapshots when checking all columns ([#2975](https://github.com/dbt-labs/dbt-core/issues/2975), [#5389](https://github.com/dbt-labs/dbt-core/pull/5389))
|
||||
- fixes handling of RESET color code with USE_COLORS=False ([#5288](https://github.com/dbt-labs/dbt-core/issues/5288), [#5394](https://github.com/dbt-labs/dbt-core/pull/5394))
|
||||
- Remove duplicate key checking introduced in 1.2.0a1 ([#5331](https://github.com/dbt-labs/dbt-core/issues/5331), [#5403](https://github.com/dbt-labs/dbt-core/pull/5403))
|
||||
- Rename try to strict for more intuitiveness ([#5475](https://github.com/dbt-labs/dbt-core/issues/5475), [#5477](https://github.com/dbt-labs/dbt-core/pull/5477))
|
||||
### Docs
|
||||
- Fixed sample SQL Code for sources when no database is defined ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
|
||||
- Add support for `file:` selector in DAG viz ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
|
||||
- [Snyk] Upgrade prismjs from 1.27.0 to 1.28.0 ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
|
||||
- Run build and tests in CI checks ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
|
||||
- Improve metrics DAG viz and documentation page ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
|
||||
- Upgrade cytoscape.js fork ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
|
||||
### Under the Hood
|
||||
- Migrating 005_simple_seed to the new test framework. ([#200](https://github.com/dbt-labs/dbt-core/issues/200), [#5013](https://github.com/dbt-labs/dbt-core/pull/5013))
|
||||
- Convert 029_docs_generate tests to new framework ([#5035](https://github.com/dbt-labs/dbt-core/issues/5035), [#5058](https://github.com/dbt-labs/dbt-core/pull/5058))
|
||||
@@ -46,24 +63,41 @@
|
||||
- Fix unit test test_graph_selection ([#5323](https://github.com/dbt-labs/dbt-core/issues/5323), [#5324](https://github.com/dbt-labs/dbt-core/pull/5324))
|
||||
- Update context readme + clean up context code" ([#4796](https://github.com/dbt-labs/dbt-core/issues/4796), [#5334](https://github.com/dbt-labs/dbt-core/pull/5334))
|
||||
- removed script meant for snowflake to snowflake ([#5361](https://github.com/dbt-labs/dbt-core/issues/5361), [#5362](https://github.com/dbt-labs/dbt-core/pull/5362))
|
||||
- Added the suggested RegEx to check the SemVer string within a package dependency and improved invalid version error handling. ([#5201](https://github.com/dbt-labs/dbt-core/issues/5201), [#5370](https://github.com/dbt-labs/dbt-core/pull/5370))
|
||||
- Add annotation to render_value method reimplemented in #5334 ([#4796](https://github.com/dbt-labs/dbt-core/issues/4796), [#5382](https://github.com/dbt-labs/dbt-core/pull/5382))
|
||||
- Bump manifest version to v6 ([#5417](https://github.com/dbt-labs/dbt-core/issues/5417), [#5430](https://github.com/dbt-labs/dbt-core/pull/5430))
|
||||
- Add tests for SQL grants ([#5437](https://github.com/dbt-labs/dbt-core/issues/5437), [#5447](https://github.com/dbt-labs/dbt-core/pull/5447))
|
||||
### Dependencies
|
||||
- Bump ubuntu from 20.04 to 22.04 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5141](https://github.com/dbt-labs/dbt-core/pull/5141))
|
||||
- Bumping hologram version ([#5219](https://github.com/dbt-labs/dbt-core/issues/5219), [#5218](https://github.com/dbt-labs/dbt-core/pull/5218))
|
||||
- Bump mypy from 0.942 to 0.961 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5337](https://github.com/dbt-labs/dbt-core/pull/5337))
|
||||
- Bump python from 3.10.3-slim-bullseye to 3.10.5-slim-bullseye in /docker ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5367](https://github.com/dbt-labs/dbt-core/pull/5367))
|
||||
- Update colorama requirement from <0.4.5,>=0.3.9 to >=0.3.9,<0.4.6 in /core ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5388](https://github.com/dbt-labs/dbt-core/pull/5388))
|
||||
- Bump black from 22.3.0 to 22.6.0 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5420](https://github.com/dbt-labs/dbt-core/pull/5420))
|
||||
### Security
|
||||
- Move string interpolation of "secret" env vars outside of Jinja context. Update "contexts" README ([#4796](https://github.com/dbt-labs/dbt-core/issues/4796), [#5334](https://github.com/dbt-labs/dbt-core/pull/5334))
|
||||
|
||||
### Contributors
|
||||
- [@GtheSheep](https://github.com/GtheSheep) ([#4893](https://github.com/dbt-labs/dbt-core/pull/4893))
|
||||
- [@NicolasPA](https://github.com/NicolasPA) ([#5211](https://github.com/dbt-labs/dbt-core/pull/5211))
|
||||
- [@adamantike](https://github.com/adamantike) ([#5207](https://github.com/dbt-labs/dbt-core/pull/5207))
|
||||
- [@alexrosenfeld10](https://github.com/alexrosenfeld10) ([#5184](https://github.com/dbt-labs/dbt-core/pull/5184))
|
||||
- [@b-per](https://github.com/b-per) ([#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
|
||||
- [@bd3dowling](https://github.com/bd3dowling) ([#5140](https://github.com/dbt-labs/dbt-core/pull/5140))
|
||||
- [@callum-mcdata](https://github.com/callum-mcdata) ([#5027](https://github.com/dbt-labs/dbt-core/pull/5027), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
|
||||
- [@danieldiamond](https://github.com/danieldiamond) ([#4827](https://github.com/dbt-labs/dbt-core/pull/4827))
|
||||
- [@darin-reify](https://github.com/darin-reify) ([#5394](https://github.com/dbt-labs/dbt-core/pull/5394))
|
||||
- [@dbeatty10](https://github.com/dbeatty10) ([#5265](https://github.com/dbt-labs/dbt-core/pull/5265), [#5077](https://github.com/dbt-labs/dbt-core/pull/5077))
|
||||
- [@dependabot[bot]](https://github.com/dependabot[bot]) ([#5141](https://github.com/dbt-labs/dbt-core/pull/5141), [#5367](https://github.com/dbt-labs/dbt-core/pull/5367))
|
||||
- [@drewbanin](https://github.com/drewbanin) ([#5027](https://github.com/dbt-labs/dbt-core/pull/5027), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
|
||||
- [@epapineau](https://github.com/epapineau) ([#4921](https://github.com/dbt-labs/dbt-core/pull/4921))
|
||||
- [@fivetran-joemarkiewicz](https://github.com/fivetran-joemarkiewicz) ([#5370](https://github.com/dbt-labs/dbt-core/pull/5370))
|
||||
- [@groodt](https://github.com/groodt) ([#5304](https://github.com/dbt-labs/dbt-core/pull/5304))
|
||||
- [@isidentical](https://github.com/isidentical) ([#5402](https://github.com/dbt-labs/dbt-core/pull/5402))
|
||||
- [@jared-rimmer](https://github.com/jared-rimmer) ([#5364](https://github.com/dbt-labs/dbt-core/pull/5364))
|
||||
- [@jeremyyeo](https://github.com/jeremyyeo) ([#5107](https://github.com/dbt-labs/dbt-core/pull/5107), [#5146](https://github.com/dbt-labs/dbt-core/pull/5146), [#5403](https://github.com/dbt-labs/dbt-core/pull/5403))
|
||||
- [@jeremyyeo](https://github.com/jeremyyeo) ([#5107](https://github.com/dbt-labs/dbt-core/pull/5107), [#5146](https://github.com/dbt-labs/dbt-core/pull/5146), [#5403](https://github.com/dbt-labs/dbt-core/pull/5403), [#5477](https://github.com/dbt-labs/dbt-core/pull/5477))
|
||||
- [@jwills](https://github.com/jwills) ([#5241](https://github.com/dbt-labs/dbt-core/pull/5241), [#5269](https://github.com/dbt-labs/dbt-core/pull/5269))
|
||||
- [@tomasfarias](https://github.com/tomasfarias) ([#5209](https://github.com/dbt-labs/dbt-core/pull/5209))
|
||||
- [@pdebelak](https://github.com/pdebelak) ([#5356](https://github.com/dbt-labs/dbt-core/pull/5356))
|
||||
- [@pquadri](https://github.com/pquadri) ([#5389](https://github.com/dbt-labs/dbt-core/pull/5389))
|
||||
- [@tomasfarias](https://github.com/tomasfarias) ([#5432](https://github.com/dbt-labs/dbt-core/pull/5432), [#5209](https://github.com/dbt-labs/dbt-core/pull/5209))
|
||||
- [@ulisesojeda](https://github.com/ulisesojeda) ([#5366](https://github.com/dbt-labs/dbt-core/pull/5366))
|
||||
- [@volkangurel](https://github.com/volkangurel) ([#5348](https://github.com/dbt-labs/dbt-core/pull/5348))
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Dependencies
|
||||
body: "Bump ubuntu from 20.04 to 22.04"
|
||||
time: 2022-04-27T19:51:28.000000-05:00
|
||||
custom:
|
||||
Author: dependabot[bot]
|
||||
Issue: "4904"
|
||||
PR: "5141"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Dependencies
|
||||
body: "Bumping hologram version"
|
||||
time: 2022-05-06T16:09:07.000000-05:00
|
||||
custom:
|
||||
Author: leahwicz
|
||||
Issue: "5219"
|
||||
PR: "5218"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Dependencies
|
||||
body: "Bump python from 3.10.3-slim-bullseye to 3.10.5-slim-bullseye in /docker"
|
||||
time: 2022-06-13T00:14:56.000000-05:00
|
||||
custom:
|
||||
Author: dependabot[bot]
|
||||
Issue: "4904"
|
||||
PR: "5367"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Add selector method when reading selector definitions
|
||||
time: 2022-04-08T11:26:10.713088+10:00
|
||||
custom:
|
||||
Author: danieldiamond
|
||||
Issue: "4821"
|
||||
PR: "4827"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Add set and zip function to contexts
|
||||
time: 2022-04-23T23:17:56.851793+12:00
|
||||
custom:
|
||||
Author: jeremyyeo
|
||||
Issue: "2345"
|
||||
PR: "5107"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Adds itertools to modules Jinja namespace
|
||||
time: 2022-04-24T13:26:55.008246+01:00
|
||||
custom:
|
||||
Author: bd3dowling
|
||||
Issue: "5130"
|
||||
PR: "5140"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: allow target as an option in profile_template.yml
|
||||
time: 2022-04-28T06:56:44.511519-04:00
|
||||
custom:
|
||||
Author: alexrosenfeld10
|
||||
Issue: "5179"
|
||||
PR: "5184"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: 'seed: Add new macro get_csv_sql'
|
||||
time: 2022-05-03T14:29:34.847959075Z
|
||||
custom:
|
||||
Author: adamantike
|
||||
Issue: "5206"
|
||||
PR: "5207"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Grants as Node Configs
|
||||
time: 2022-05-10T20:49:49.197999-04:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "5189"
|
||||
PR: "5230"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Adds file selectors and support for file selectors in the default method selector
|
||||
time: 2022-05-12T21:57:48.289674-07:00
|
||||
custom:
|
||||
Author: jwills
|
||||
Issue: "5240"
|
||||
PR: "5241"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Move cross-db macros from dbt-utils into dbt-core global project
|
||||
time: 2022-05-18T11:46:04.557104+02:00
|
||||
custom:
|
||||
Author: jtcohen6 dbeatty10
|
||||
Issue: "4813"
|
||||
PR: "5265"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Prettify duration message at the end of execution
|
||||
time: 2022-06-11T16:39:31.725960083+01:00
|
||||
custom:
|
||||
Author: jared-rimmer
|
||||
Issue: "5253"
|
||||
PR: "5364"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Early return from dbt init if no available adapters
|
||||
time: 2022-06-14T08:20:51.096872718+02:00
|
||||
custom:
|
||||
Author: ulisesojeda
|
||||
Issue: "5365"
|
||||
PR: "5366"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Adding new cols to check_cols in snapshots
|
||||
time: 2022-03-17T21:09:16.977086+01:00
|
||||
custom:
|
||||
Author: GtheSheep
|
||||
Issue: "3146"
|
||||
PR: "4893"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Truncate relation names when appending a suffix that will result in len > 63
|
||||
characters using make_temp_relation and make_backup_relation macros
|
||||
time: 2022-03-22T17:37:53.320082-07:00
|
||||
custom:
|
||||
Author: epapineau
|
||||
Issue: "2869"
|
||||
PR: "4921"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Restore ability to utilize `updated_at` for check_cols snapshots
|
||||
time: 2022-04-15T11:29:27.063462-06:00
|
||||
custom:
|
||||
Author: dbeatty10
|
||||
Issue: "5076"
|
||||
PR: "5077"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Fix retry logic to return values after initial try
|
||||
time: 2022-04-22T13:12:27.239055-05:00
|
||||
custom:
|
||||
Author: emmyoop
|
||||
Issue: "5023"
|
||||
PR: "5137"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Use yaml renderer (with target context) for rendering selectors
|
||||
time: 2022-04-22T13:56:45.147893-04:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "5131"
|
||||
PR: "5136"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Scrub secret env vars from CommandError in exception stacktrace
|
||||
time: 2022-04-25T20:39:24.365495+02:00
|
||||
custom:
|
||||
Author: jtcohen6
|
||||
Issue: "5151"
|
||||
PR: "5152"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Ensure the metric name does not contain spaces
|
||||
time: 2022-04-26T20:21:04.360693-04:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "4572"
|
||||
PR: "5173"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: When parsing 'all_sources' should be a list of unique dirs
|
||||
time: 2022-04-27T10:26:48.648388-04:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "5120"
|
||||
PR: "5176"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Add warning if yaml contains duplicate keys
|
||||
time: 2022-04-28T10:01:57.893956+12:00
|
||||
custom:
|
||||
Author: jeremyyeo
|
||||
Issue: "5114"
|
||||
PR: "5146"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Modifying the drop_test_schema to work better with Redshift issues around locked
|
||||
tables and current transactions
|
||||
time: 2022-04-29T16:07:42.750046-05:00
|
||||
custom:
|
||||
Author: Mcknight-42
|
||||
Issue: "5200"
|
||||
PR: "5198"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Fix column comparison in snapshot_check_all_get_existing_columns for check-strategy
|
||||
snapshots with explicit check_cols defined
|
||||
time: 2022-05-09T13:00:21.649028+02:00
|
||||
custom:
|
||||
Author: jtcohen6
|
||||
Issue: "5222"
|
||||
PR: "5223"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Changed how `--select state:modified` detects changes for macros nodes depend
|
||||
on
|
||||
time: 2022-05-09T13:13:12.889074-05:00
|
||||
custom:
|
||||
Author: stu-k
|
||||
Issue: "5202"
|
||||
PR: "5224"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Fix column comparison in snapshot_check_all_get_existing_columns to use adapter.get_columns_in_relation
|
||||
time: 2022-05-11T12:32:38.313321+02:00
|
||||
custom:
|
||||
Author: jtcohen6
|
||||
Issue: "5222"
|
||||
PR: "5232"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Remove docs file from manifest when removing doc node
|
||||
time: 2022-05-18T13:46:10.167143-04:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "4146"
|
||||
PR: "5270"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Remove duplicate dbt script entry
|
||||
time: 2022-06-01T08:13:14.067001+10:00
|
||||
custom:
|
||||
Author: groodt
|
||||
Issue: "5314"
|
||||
PR: "5304"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Change node ancestor/descendant algo, fixes issue where downstream models aren't
|
||||
run when using networkx >= 2.8.1
|
||||
time: 2022-06-01T13:59:08.886215-05:00
|
||||
custom:
|
||||
Author: iknox-fa
|
||||
Issue: "5286"
|
||||
PR: "5326"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Fixing Windows color regression
|
||||
time: 2022-06-01T19:42:34.263009-04:00
|
||||
custom:
|
||||
Author: leahwicz
|
||||
Issue: "5191"
|
||||
PR: "5327"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Fixes
|
||||
body: 'Define compatibility for older manifest versions when using state: selection
|
||||
methods'
|
||||
time: 2022-06-08T08:09:14.321735+02:00
|
||||
custom:
|
||||
Author: jtcohen6
|
||||
Issue: "5213"
|
||||
PR: "5346"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Remove duplicate key checking introduced in 1.2.0a1
|
||||
time: 2022-06-23T08:24:31.900647+12:00
|
||||
custom:
|
||||
Author: jeremyyeo
|
||||
Issue: "5331"
|
||||
PR: "5403"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Migrating 005_simple_seed to the new test framework.
|
||||
time: 2022-04-09T04:05:39.20045-07:00
|
||||
custom:
|
||||
Author: versusfacit
|
||||
Issue: "200"
|
||||
PR: "5013"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Convert 029_docs_generate tests to new framework
|
||||
time: 2022-04-13T18:30:14.706391-04:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "5035"
|
||||
PR: "5058"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Move package deprecation check outside of package cache
|
||||
time: 2022-04-14T13:22:06.157579-05:00
|
||||
custom:
|
||||
Author: emmyoop
|
||||
Issue: "5068"
|
||||
PR: "5069"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: removal of scaffold first attempt and create_adapter_plugin.py as they are deprecated new scaffold can be found https://github.com/dbt-labs/dbt-database-adapter-scaffold
|
||||
time: 2022-04-20T12:00:25.171923-05:00
|
||||
custom:
|
||||
Author: McKnight-42
|
||||
Issue: "4980"
|
||||
PR: "5117"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Mypy -> 0.942 + fixed import logic to allow for full mypy coverage
|
||||
time: 2022-04-27T11:21:27.499359-05:00
|
||||
custom:
|
||||
Author: iknox-fa
|
||||
Issue: "4805"
|
||||
PR: "5171"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Converted dbt list tests to pytest
|
||||
time: 2022-04-27T14:06:28.882908-05:00
|
||||
custom:
|
||||
Author: stu-k
|
||||
Issue: "5049"
|
||||
PR: "5178"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: 'Fix: Call str and repr for UnsetProfileConfig without a RuntimeException'
|
||||
time: 2022-05-03T19:52:12.793729384+02:00
|
||||
custom:
|
||||
Author: tomasfarias
|
||||
Issue: "5081"
|
||||
PR: "5209"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Improve tracking error logging message
|
||||
time: 2022-05-04T01:00:31.60387036+02:00
|
||||
custom:
|
||||
Author: NicolasPA
|
||||
Issue: "5197"
|
||||
PR: "5211"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: 'Clean up materialization logic: more consistent relation names, loading from
|
||||
cache'
|
||||
time: 2022-05-09T09:26:28.551068+02:00
|
||||
custom:
|
||||
Author: jtcohen6
|
||||
Issue: "2869"
|
||||
PR: "4921"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Use the default Python version for local dev and test instead of requiring Python
|
||||
3.8
|
||||
time: 2022-05-18T09:51:44.603193-07:00
|
||||
custom:
|
||||
Author: jwills
|
||||
Issue: "5257"
|
||||
PR: "5269"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Fix test for context set function
|
||||
time: 2022-05-18T14:55:22.554316-04:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "5266"
|
||||
PR: "5272"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Fix pip upgrade step in CI for Windows
|
||||
time: 2022-06-01T10:52:45.872931-04:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "5321"
|
||||
PR: "5320"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Fix unit test test_graph_selection
|
||||
time: 2022-06-01T11:26:48.725831-04:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "5323"
|
||||
PR: "5324"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Update context readme + clean up context code"
|
||||
time: 2022-06-06T23:03:53.022568+02:00
|
||||
custom:
|
||||
Author: jtcohen6
|
||||
Issue: "4796"
|
||||
PR: "5334"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: removed script meant for snowflake to snowflake
|
||||
time: 2022-06-10T11:10:56.080236-05:00
|
||||
custom:
|
||||
Author: McKnight-42
|
||||
Issue: "5361"
|
||||
PR: "5362"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Dependencies
|
||||
body: "Bump mypy from 0.942 to 0.961"
|
||||
time: 2022-06-07T00:09:25.000000-05:00
|
||||
custom:
|
||||
Author: dependabot[bot]
|
||||
Issue: "4904"
|
||||
PR: "5337"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Dependencies
|
||||
body: "Update colorama requirement from <0.4.5,>=0.3.9 to >=0.3.9,<0.4.6 in /core"
|
||||
time: 2022-06-17T00:18:49.000000-05:00
|
||||
custom:
|
||||
Author: dependabot[bot]
|
||||
Issue: "4904"
|
||||
PR: "5388"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Dependencies
|
||||
body: "Bump black from 22.3.0 to 22.6.0"
|
||||
time: 2022-06-29T00:08:31.000000-05:00
|
||||
custom:
|
||||
Author: dependabot[bot]
|
||||
Issue: "4904"
|
||||
PR: "5420"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Docs
|
||||
body: Fixed sample SQL Code for sources when no database is defined
|
||||
time: 2022-07-06T14:20:29.7347-05:00
|
||||
custom:
|
||||
Author: b-per
|
||||
Issue: "5255"
|
||||
PR: "5446"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Docs
|
||||
body: Add support for `file:` selector in DAG viz
|
||||
time: 2022-07-06T14:21:42.544814-05:00
|
||||
custom:
|
||||
Author: drewbanin
|
||||
Issue: "5255"
|
||||
PR: "5446"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Docs
|
||||
body: '[Snyk] Upgrade prismjs from 1.27.0 to 1.28.0'
|
||||
time: 2022-07-06T14:22:26.039137-05:00
|
||||
custom:
|
||||
Author: emmyoop
|
||||
Issue: "5255"
|
||||
PR: "5446"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Docs
|
||||
body: Run build and tests in CI checks
|
||||
time: 2022-07-06T14:23:00.842859-05:00
|
||||
custom:
|
||||
Author: drewbanin
|
||||
Issue: "5255"
|
||||
PR: "5446"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Docs
|
||||
body: Improve metrics DAG viz and documentation page
|
||||
time: 2022-07-06T14:24:10.120545-05:00
|
||||
custom:
|
||||
Author: drewbanin callum-mcdata
|
||||
Issue: "5255"
|
||||
PR: "5446"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Docs
|
||||
body: Upgrade cytoscape.js fork
|
||||
time: 2022-07-06T14:25:05.575755-05:00
|
||||
custom:
|
||||
Author: emmyoop
|
||||
Issue: "5255"
|
||||
PR: "5446"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Features
|
||||
body: 'Allow customizing `target-path` and `log-path` through
|
||||
environment variables and CLI flags.'
|
||||
time: 2022-06-22T19:23:29.748577478+03:00
|
||||
custom:
|
||||
Author: isidentical
|
||||
Issue: "5399"
|
||||
PR: "5402"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Move type_* macros from dbt-utils into dbt-core, with tests
|
||||
time: 2022-06-30T13:54:52.165139+02:00
|
||||
custom:
|
||||
Author: jtcohen6
|
||||
Issue: "5317"
|
||||
PR: "5428"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Add support for ratio metrics
|
||||
time: 2022-07-05T08:30:26.494837-05:00
|
||||
custom:
|
||||
Author: drewbanin callum-mcdata
|
||||
Issue: "4884"
|
||||
PR: "5027"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Features
|
||||
body: Allow users to define grants as a reasonable default in the dbt_project.yml
|
||||
or within each model sql or yml file combined.
|
||||
time: 2022-07-11T11:15:14.695386-05:00
|
||||
custom:
|
||||
Author: McKnight-42
|
||||
Issue: "5263"
|
||||
PR: "5369"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Add inheritance to materialization macro resolution
|
||||
time: 2022-06-08T10:42:51.839945-04:00
|
||||
custom:
|
||||
Author: volkangurel
|
||||
Issue: "4646"
|
||||
PR: "5348"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Improve pluralizations for Documentation and SqlOperation NodeTypes
|
||||
time: 2022-06-09T11:52:46.578469-05:00
|
||||
custom:
|
||||
Author: pdebelak
|
||||
Issue: "5352"
|
||||
PR: "5356"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Properly use quotes for Snowflake snapshots when checking all columns
|
||||
time: 2022-06-17T11:44:43.978834+02:00
|
||||
custom:
|
||||
Author: pquadri
|
||||
Issue: "2975"
|
||||
PR: "5389"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: fixes handling of RESET color code with USE_COLORS=False
|
||||
time: 2022-06-17T16:00:27.038058-04:00
|
||||
custom:
|
||||
Author: darin-reify
|
||||
Issue: "5288"
|
||||
PR: "5394"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Security
|
||||
body: Move string interpolation of "secret" env vars outside of Jinja context. Update "contexts" README
|
||||
time: 2022-06-06T23:03:53.022568+02:00
|
||||
custom:
|
||||
Author: jtcohen6
|
||||
Issue: "4796"
|
||||
PR: "5334"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Added the suggested RegEx to check the SemVer string within a package dependency
|
||||
and improved invalid version error handling.
|
||||
time: 2022-06-13T17:07:03.773668-05:00
|
||||
custom:
|
||||
Author: fivetran-joemarkiewicz
|
||||
Issue: "5201"
|
||||
PR: "5370"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: 'Add annotation to render_value method reimplemented in #5334'
|
||||
time: 2022-06-16T12:05:16.474078+02:00
|
||||
custom:
|
||||
Author: jtcohen6
|
||||
Issue: "4796"
|
||||
PR: "5382"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Bump manifest version to v6
|
||||
time: 2022-07-05T14:21:20.66768-04:00
|
||||
custom:
|
||||
Author: leahwicz
|
||||
Issue: "5417"
|
||||
PR: "5430"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Add tests for SQL grants
|
||||
time: 2022-07-06T21:50:01.498562-04:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "5437"
|
||||
PR: "5447"
|
||||
42
CHANGELOG.md
42
CHANGELOG.md
@@ -6,7 +6,7 @@
|
||||
- Do not edit this file directly. This file is auto-generated using [changie](https://github.com/miniscruff/changie). For details on how to document a change, see [the contributing guide](https://github.com/dbt-labs/dbt-core/blob/main/CONTRIBUTING.md#adding-changelog-entry)
|
||||
|
||||
|
||||
## dbt-core 1.2.0-b1 - June 24, 2022
|
||||
## dbt-core 1.2.0 - July 26, 2022
|
||||
### Features
|
||||
- Add selector method when reading selector definitions ([#4821](https://github.com/dbt-labs/dbt-core/issues/4821), [#4827](https://github.com/dbt-labs/dbt-core/pull/4827))
|
||||
- Add set and zip function to contexts ([#2345](https://github.com/dbt-labs/dbt-core/issues/2345), [#5107](https://github.com/dbt-labs/dbt-core/pull/5107))
|
||||
@@ -18,6 +18,11 @@
|
||||
- Move cross-db macros from dbt-utils into dbt-core global project ([#4813](https://github.com/dbt-labs/dbt-core/issues/4813), [#5265](https://github.com/dbt-labs/dbt-core/pull/5265))
|
||||
- Prettify duration message at the end of execution ([#5253](https://github.com/dbt-labs/dbt-core/issues/5253), [#5364](https://github.com/dbt-labs/dbt-core/pull/5364))
|
||||
- Early return from dbt init if no available adapters ([#5365](https://github.com/dbt-labs/dbt-core/issues/5365), [#5366](https://github.com/dbt-labs/dbt-core/pull/5366))
|
||||
- Allow customizing `target-path` and `log-path` through environment variables and CLI flags. ([#5399](https://github.com/dbt-labs/dbt-core/issues/5399), [#5402](https://github.com/dbt-labs/dbt-core/pull/5402))
|
||||
- Move type_* macros from dbt-utils into dbt-core, with tests ([#5317](https://github.com/dbt-labs/dbt-core/issues/5317), [#5428](https://github.com/dbt-labs/dbt-core/pull/5428))
|
||||
- Add support for ratio metrics ([#4884](https://github.com/dbt-labs/dbt-core/issues/4884), [#5027](https://github.com/dbt-labs/dbt-core/pull/5027))
|
||||
- Allow users to define grants as a reasonable default in the dbt_project.yml or within each model sql or yml file combined. ([#5263](https://github.com/dbt-labs/dbt-core/issues/5263), [#5369](https://github.com/dbt-labs/dbt-core/pull/5369))
|
||||
- Add reusable function for retrying adapter connections. Utilize said function to add retries for Postgres (and Redshift). ([#5022](https://github.com/dbt-labs/dbt-core/issues/5022), [#5432](https://github.com/dbt-labs/dbt-core/pull/5432))
|
||||
### Fixes
|
||||
- Adding new cols to check_cols in snapshots ([#3146](https://github.com/dbt-labs/dbt-core/issues/3146), [#4893](https://github.com/dbt-labs/dbt-core/pull/4893))
|
||||
- Truncate relation names when appending a suffix that will result in len > 63 characters using make_temp_relation and make_backup_relation macros ([#2869](https://github.com/dbt-labs/dbt-core/issues/2869), [#4921](https://github.com/dbt-labs/dbt-core/pull/4921))
|
||||
@@ -37,7 +42,19 @@
|
||||
- Change node ancestor/descendant algo, fixes issue where downstream models aren't run when using networkx >= 2.8.1 ([#5286](https://github.com/dbt-labs/dbt-core/issues/5286), [#5326](https://github.com/dbt-labs/dbt-core/pull/5326))
|
||||
- Fixing Windows color regression ([#5191](https://github.com/dbt-labs/dbt-core/issues/5191), [#5327](https://github.com/dbt-labs/dbt-core/pull/5327))
|
||||
- Define compatibility for older manifest versions when using state: selection methods ([#5213](https://github.com/dbt-labs/dbt-core/issues/5213), [#5346](https://github.com/dbt-labs/dbt-core/pull/5346))
|
||||
- Add inheritance to materialization macro resolution ([#4646](https://github.com/dbt-labs/dbt-core/issues/4646), [#5348](https://github.com/dbt-labs/dbt-core/pull/5348))
|
||||
- Improve pluralizations for Documentation and SqlOperation NodeTypes ([#5352](https://github.com/dbt-labs/dbt-core/issues/5352), [#5356](https://github.com/dbt-labs/dbt-core/pull/5356))
|
||||
- Properly use quotes for Snowflake snapshots when checking all columns ([#2975](https://github.com/dbt-labs/dbt-core/issues/2975), [#5389](https://github.com/dbt-labs/dbt-core/pull/5389))
|
||||
- fixes handling of RESET color code with USE_COLORS=False ([#5288](https://github.com/dbt-labs/dbt-core/issues/5288), [#5394](https://github.com/dbt-labs/dbt-core/pull/5394))
|
||||
- Remove duplicate key checking introduced in 1.2.0a1 ([#5331](https://github.com/dbt-labs/dbt-core/issues/5331), [#5403](https://github.com/dbt-labs/dbt-core/pull/5403))
|
||||
- Rename try to strict for more intuitiveness ([#5475](https://github.com/dbt-labs/dbt-core/issues/5475), [#5477](https://github.com/dbt-labs/dbt-core/pull/5477))
|
||||
### Docs
|
||||
- Fixed sample SQL Code for sources when no database is defined ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
|
||||
- Add support for `file:` selector in DAG viz ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
|
||||
- [Snyk] Upgrade prismjs from 1.27.0 to 1.28.0 ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
|
||||
- Run build and tests in CI checks ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
|
||||
- Improve metrics DAG viz and documentation page ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
|
||||
- Upgrade cytoscape.js fork ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
|
||||
### Under the Hood
|
||||
- Migrating 005_simple_seed to the new test framework. ([#200](https://github.com/dbt-labs/dbt-core/issues/200), [#5013](https://github.com/dbt-labs/dbt-core/pull/5013))
|
||||
- Convert 029_docs_generate tests to new framework ([#5035](https://github.com/dbt-labs/dbt-core/issues/5035), [#5058](https://github.com/dbt-labs/dbt-core/pull/5058))
|
||||
@@ -54,27 +71,44 @@
|
||||
- Fix unit test test_graph_selection ([#5323](https://github.com/dbt-labs/dbt-core/issues/5323), [#5324](https://github.com/dbt-labs/dbt-core/pull/5324))
|
||||
- Update context readme + clean up context code" ([#4796](https://github.com/dbt-labs/dbt-core/issues/4796), [#5334](https://github.com/dbt-labs/dbt-core/pull/5334))
|
||||
- removed script meant for snowflake to snowflake ([#5361](https://github.com/dbt-labs/dbt-core/issues/5361), [#5362](https://github.com/dbt-labs/dbt-core/pull/5362))
|
||||
- Added the suggested RegEx to check the SemVer string within a package dependency and improved invalid version error handling. ([#5201](https://github.com/dbt-labs/dbt-core/issues/5201), [#5370](https://github.com/dbt-labs/dbt-core/pull/5370))
|
||||
- Add annotation to render_value method reimplemented in #5334 ([#4796](https://github.com/dbt-labs/dbt-core/issues/4796), [#5382](https://github.com/dbt-labs/dbt-core/pull/5382))
|
||||
- Bump manifest version to v6 ([#5417](https://github.com/dbt-labs/dbt-core/issues/5417), [#5430](https://github.com/dbt-labs/dbt-core/pull/5430))
|
||||
- Add tests for SQL grants ([#5437](https://github.com/dbt-labs/dbt-core/issues/5437), [#5447](https://github.com/dbt-labs/dbt-core/pull/5447))
|
||||
### Dependencies
|
||||
- Bump ubuntu from 20.04 to 22.04 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5141](https://github.com/dbt-labs/dbt-core/pull/5141))
|
||||
- Bumping hologram version ([#5219](https://github.com/dbt-labs/dbt-core/issues/5219), [#5218](https://github.com/dbt-labs/dbt-core/pull/5218))
|
||||
- Bump mypy from 0.942 to 0.961 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5337](https://github.com/dbt-labs/dbt-core/pull/5337))
|
||||
- Bump python from 3.10.3-slim-bullseye to 3.10.5-slim-bullseye in /docker ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5367](https://github.com/dbt-labs/dbt-core/pull/5367))
|
||||
- Update colorama requirement from <0.4.5,>=0.3.9 to >=0.3.9,<0.4.6 in /core ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5388](https://github.com/dbt-labs/dbt-core/pull/5388))
|
||||
- Bump black from 22.3.0 to 22.6.0 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5420](https://github.com/dbt-labs/dbt-core/pull/5420))
|
||||
### Security
|
||||
- Move string interpolation of "secret" env vars outside of Jinja context. Update "contexts" README ([#4796](https://github.com/dbt-labs/dbt-core/issues/4796), [#5334](https://github.com/dbt-labs/dbt-core/pull/5334))
|
||||
|
||||
### Contributors
|
||||
- [@GtheSheep](https://github.com/GtheSheep) ([#4893](https://github.com/dbt-labs/dbt-core/pull/4893))
|
||||
- [@NicolasPA](https://github.com/NicolasPA) ([#5211](https://github.com/dbt-labs/dbt-core/pull/5211))
|
||||
- [@adamantike](https://github.com/adamantike) ([#5207](https://github.com/dbt-labs/dbt-core/pull/5207))
|
||||
- [@alexrosenfeld10](https://github.com/alexrosenfeld10) ([#5184](https://github.com/dbt-labs/dbt-core/pull/5184))
|
||||
- [@b-per](https://github.com/b-per) ([#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
|
||||
- [@bd3dowling](https://github.com/bd3dowling) ([#5140](https://github.com/dbt-labs/dbt-core/pull/5140))
|
||||
- [@callum-mcdata](https://github.com/callum-mcdata) ([#5027](https://github.com/dbt-labs/dbt-core/pull/5027), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
|
||||
- [@danieldiamond](https://github.com/danieldiamond) ([#4827](https://github.com/dbt-labs/dbt-core/pull/4827))
|
||||
- [@darin-reify](https://github.com/darin-reify) ([#5394](https://github.com/dbt-labs/dbt-core/pull/5394))
|
||||
- [@dbeatty10](https://github.com/dbeatty10) ([#5265](https://github.com/dbt-labs/dbt-core/pull/5265), [#5077](https://github.com/dbt-labs/dbt-core/pull/5077))
|
||||
- [@dependabot[bot]](https://github.com/dependabot[bot]) ([#5141](https://github.com/dbt-labs/dbt-core/pull/5141), [#5367](https://github.com/dbt-labs/dbt-core/pull/5367))
|
||||
- [@drewbanin](https://github.com/drewbanin) ([#5027](https://github.com/dbt-labs/dbt-core/pull/5027), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
|
||||
- [@epapineau](https://github.com/epapineau) ([#4921](https://github.com/dbt-labs/dbt-core/pull/4921))
|
||||
- [@fivetran-joemarkiewicz](https://github.com/fivetran-joemarkiewicz) ([#5370](https://github.com/dbt-labs/dbt-core/pull/5370))
|
||||
- [@groodt](https://github.com/groodt) ([#5304](https://github.com/dbt-labs/dbt-core/pull/5304))
|
||||
- [@isidentical](https://github.com/isidentical) ([#5402](https://github.com/dbt-labs/dbt-core/pull/5402))
|
||||
- [@jared-rimmer](https://github.com/jared-rimmer) ([#5364](https://github.com/dbt-labs/dbt-core/pull/5364))
|
||||
- [@jeremyyeo](https://github.com/jeremyyeo) ([#5107](https://github.com/dbt-labs/dbt-core/pull/5107), [#5146](https://github.com/dbt-labs/dbt-core/pull/5146), [#5403](https://github.com/dbt-labs/dbt-core/pull/5403))
|
||||
- [@jeremyyeo](https://github.com/jeremyyeo) ([#5107](https://github.com/dbt-labs/dbt-core/pull/5107), [#5146](https://github.com/dbt-labs/dbt-core/pull/5146), [#5403](https://github.com/dbt-labs/dbt-core/pull/5403), [#5477](https://github.com/dbt-labs/dbt-core/pull/5477))
|
||||
- [@jwills](https://github.com/jwills) ([#5241](https://github.com/dbt-labs/dbt-core/pull/5241), [#5269](https://github.com/dbt-labs/dbt-core/pull/5269))
|
||||
- [@tomasfarias](https://github.com/tomasfarias) ([#5209](https://github.com/dbt-labs/dbt-core/pull/5209))
|
||||
- [@pdebelak](https://github.com/pdebelak) ([#5356](https://github.com/dbt-labs/dbt-core/pull/5356))
|
||||
- [@pquadri](https://github.com/pquadri) ([#5389](https://github.com/dbt-labs/dbt-core/pull/5389))
|
||||
- [@tomasfarias](https://github.com/tomasfarias) ([#5432](https://github.com/dbt-labs/dbt-core/pull/5432), [#5209](https://github.com/dbt-labs/dbt-core/pull/5209))
|
||||
- [@ulisesojeda](https://github.com/ulisesojeda) ([#5366](https://github.com/dbt-labs/dbt-core/pull/5366))
|
||||
- [@volkangurel](https://github.com/volkangurel) ([#5348](https://github.com/dbt-labs/dbt-core/pull/5348))
|
||||
|
||||
|
||||
## Previous Releases
|
||||
|
||||
@@ -1,10 +1,24 @@
|
||||
import abc
|
||||
import os
|
||||
from time import sleep
|
||||
import sys
|
||||
|
||||
# multiprocessing.RLock is a function returning this type
|
||||
from multiprocessing.synchronize import RLock
|
||||
from threading import get_ident
|
||||
from typing import Dict, Tuple, Hashable, Optional, ContextManager, List
|
||||
from typing import (
|
||||
Any,
|
||||
Dict,
|
||||
Tuple,
|
||||
Hashable,
|
||||
Optional,
|
||||
ContextManager,
|
||||
List,
|
||||
Type,
|
||||
Union,
|
||||
Iterable,
|
||||
Callable,
|
||||
)
|
||||
|
||||
import agate
|
||||
|
||||
@@ -21,6 +35,7 @@ from dbt.contracts.graph.manifest import Manifest
|
||||
from dbt.adapters.base.query_headers import (
|
||||
MacroQueryStringSetter,
|
||||
)
|
||||
from dbt.events import AdapterLogger
|
||||
from dbt.events.functions import fire_event
|
||||
from dbt.events.types import (
|
||||
NewConnection,
|
||||
@@ -34,6 +49,9 @@ from dbt.events.types import (
|
||||
)
|
||||
from dbt import flags
|
||||
|
||||
SleepTime = Union[int, float] # As taken by time.sleep.
|
||||
AdapterHandle = Any # Adapter connection handle objects can be any class.
|
||||
|
||||
|
||||
class BaseConnectionManager(metaclass=abc.ABCMeta):
|
||||
"""Methods to implement:
|
||||
@@ -159,6 +177,94 @@ class BaseConnectionManager(metaclass=abc.ABCMeta):
|
||||
conn.name = conn_name
|
||||
return conn
|
||||
|
||||
@classmethod
|
||||
def retry_connection(
|
||||
cls,
|
||||
connection: Connection,
|
||||
connect: Callable[[], AdapterHandle],
|
||||
logger: AdapterLogger,
|
||||
retryable_exceptions: Iterable[Type[Exception]],
|
||||
retry_limit: int = 1,
|
||||
retry_timeout: Union[Callable[[int], SleepTime], SleepTime] = 1,
|
||||
_attempts: int = 0,
|
||||
) -> Connection:
|
||||
"""Given a Connection, set its handle by calling connect.
|
||||
|
||||
The calls to connect will be retried up to retry_limit times to deal with transient
|
||||
connection errors. By default, one retry will be attempted if retryable_exceptions is set.
|
||||
|
||||
:param Connection connection: An instance of a Connection that needs a handle to be set,
|
||||
usually when attempting to open it.
|
||||
:param connect: A callable that returns the appropiate connection handle for a
|
||||
given adapter. This callable will be retried retry_limit times if a subclass of any
|
||||
Exception in retryable_exceptions is raised by connect.
|
||||
:type connect: Callable[[], AdapterHandle]
|
||||
:param AdapterLogger logger: A logger to emit messages on retry attempts or errors. When
|
||||
handling expected errors, we call debug, and call warning on unexpected errors or when
|
||||
all retry attempts have been exhausted.
|
||||
:param retryable_exceptions: An iterable of exception classes that if raised by
|
||||
connect should trigger a retry.
|
||||
:type retryable_exceptions: Iterable[Type[Exception]]
|
||||
:param int retry_limit: How many times to retry the call to connect. If this limit
|
||||
is exceeded before a successful call, a FailedToConnectException will be raised.
|
||||
Must be non-negative.
|
||||
:param retry_timeout: Time to wait between attempts to connect. Can also take a
|
||||
Callable that takes the number of attempts so far, beginning at 0, and returns an int
|
||||
or float to be passed to time.sleep.
|
||||
:type retry_timeout: Union[Callable[[int], SleepTime], SleepTime] = 1
|
||||
:param int _attempts: Parameter used to keep track of the number of attempts in calling the
|
||||
connect function across recursive calls. Passed as an argument to retry_timeout if it
|
||||
is a Callable. This parameter should not be set by the initial caller.
|
||||
:raises dbt.exceptions.FailedToConnectException: Upon exhausting all retry attempts without
|
||||
successfully acquiring a handle.
|
||||
:return: The given connection with its appropriate state and handle attributes set
|
||||
depending on whether we successfully acquired a handle or not.
|
||||
"""
|
||||
timeout = retry_timeout(_attempts) if callable(retry_timeout) else retry_timeout
|
||||
if timeout < 0:
|
||||
raise dbt.exceptions.FailedToConnectException(
|
||||
"retry_timeout cannot be negative or return a negative time."
|
||||
)
|
||||
|
||||
if retry_limit < 0 or retry_limit > sys.getrecursionlimit():
|
||||
# This guard is not perfect others may add to the recursion limit (e.g. built-ins).
|
||||
connection.handle = None
|
||||
connection.state = ConnectionState.FAIL
|
||||
raise dbt.exceptions.FailedToConnectException("retry_limit cannot be negative")
|
||||
|
||||
try:
|
||||
connection.handle = connect()
|
||||
connection.state = ConnectionState.OPEN
|
||||
return connection
|
||||
|
||||
except tuple(retryable_exceptions) as e:
|
||||
if retry_limit <= 0:
|
||||
connection.handle = None
|
||||
connection.state = ConnectionState.FAIL
|
||||
raise dbt.exceptions.FailedToConnectException(str(e))
|
||||
|
||||
logger.debug(
|
||||
f"Got a retryable error when attempting to open a {cls.TYPE} connection.\n"
|
||||
f"{retry_limit} attempts remaining. Retrying in {timeout} seconds.\n"
|
||||
f"Error:\n{e}"
|
||||
)
|
||||
|
||||
sleep(timeout)
|
||||
return cls.retry_connection(
|
||||
connection=connection,
|
||||
connect=connect,
|
||||
logger=logger,
|
||||
retry_limit=retry_limit - 1,
|
||||
retry_timeout=retry_timeout,
|
||||
retryable_exceptions=retryable_exceptions,
|
||||
_attempts=_attempts + 1,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
connection.handle = None
|
||||
connection.state = ConnectionState.FAIL
|
||||
raise dbt.exceptions.FailedToConnectException(str(e))
|
||||
|
||||
@abc.abstractmethod
|
||||
def cancel_open(self) -> Optional[List[str]]:
|
||||
"""Cancel all open connections on the adapter. (passable)"""
|
||||
|
||||
@@ -474,19 +474,17 @@ class BaseContext(metaclass=ContextMeta):
|
||||
|
||||
@contextmember
|
||||
@staticmethod
|
||||
def try_set(value: Iterable[Any]) -> Set[Any]:
|
||||
"""The `try_set` context method can be used to convert any iterable
|
||||
def set_strict(value: Iterable[Any]) -> Set[Any]:
|
||||
"""The `set_strict` context method can be used to convert any iterable
|
||||
to a sequence of iterable elements that are unique (a set). The
|
||||
difference to the `set` context method is that the `try_set` method
|
||||
difference to the `set` context method is that the `set_strict` method
|
||||
will raise an exception on a TypeError.
|
||||
|
||||
:param value: The iterable
|
||||
:param default: A default value to return if the `value` argument
|
||||
is not an iterable
|
||||
|
||||
Usage:
|
||||
{% set my_list = [1, 2, 2, 3] %}
|
||||
{% set my_set = try_set(my_list) %}
|
||||
{% set my_set = set_strict(my_list) %}
|
||||
{% do log(my_set) %} {# {1, 2, 3} #}
|
||||
"""
|
||||
try:
|
||||
@@ -497,7 +495,7 @@ class BaseContext(metaclass=ContextMeta):
|
||||
@contextmember("zip")
|
||||
@staticmethod
|
||||
def _zip(*args: Iterable[Any], default: Any = None) -> Optional[Iterable[Any]]:
|
||||
"""The `try_zip` context method can be used to used to return
|
||||
"""The `zip` context method can be used to used to return
|
||||
an iterator of tuples, where the i-th tuple contains the i-th
|
||||
element from each of the argument iterables.
|
||||
|
||||
@@ -518,21 +516,19 @@ class BaseContext(metaclass=ContextMeta):
|
||||
|
||||
@contextmember
|
||||
@staticmethod
|
||||
def try_zip(*args: Iterable[Any]) -> Iterable[Any]:
|
||||
"""The `try_zip` context method can be used to used to return
|
||||
def zip_strict(*args: Iterable[Any]) -> Iterable[Any]:
|
||||
"""The `zip_strict` context method can be used to used to return
|
||||
an iterator of tuples, where the i-th tuple contains the i-th
|
||||
element from each of the argument iterables. The difference to the
|
||||
`zip` context method is that the `try_zip` method will raise an
|
||||
`zip` context method is that the `zip_strict` method will raise an
|
||||
exception on a TypeError.
|
||||
|
||||
:param *args: Any number of iterables
|
||||
:param default: A default value to return if `*args` is not
|
||||
iterable
|
||||
|
||||
Usage:
|
||||
{% set my_list_a = [1, 2] %}
|
||||
{% set my_list_b = ['alice', 'bob'] %}
|
||||
{% set my_zip = try_zip(my_list_a, my_list_b) | list %}
|
||||
{% set my_zip = zip_strict(my_list_a, my_list_b) | list %}
|
||||
{% do log(my_set) %} {# [(1, 'alice'), (2, 'bob')] #}
|
||||
"""
|
||||
try:
|
||||
|
||||
@@ -235,5 +235,5 @@ def _get_adapter_plugin_names() -> Iterator[str]:
|
||||
yield plugin_name
|
||||
|
||||
|
||||
__version__ = "1.2.0b1"
|
||||
__version__ = "1.2.0"
|
||||
installed = get_installed_version()
|
||||
|
||||
@@ -25,7 +25,7 @@ with open(os.path.join(this_directory, "README.md")) as f:
|
||||
|
||||
|
||||
package_name = "dbt-core"
|
||||
package_version = "1.2.0b1"
|
||||
package_version = "1.2.0"
|
||||
description = """With dbt, data analysts and engineers can build analytics \
|
||||
the way engineers build applications."""
|
||||
|
||||
|
||||
@@ -14,12 +14,12 @@ FROM --platform=$build_for python:3.10.5-slim-bullseye as base
|
||||
# N.B. The refs updated automagically every release via bumpversion
|
||||
# N.B. dbt-postgres is currently found in the core codebase so a value of dbt-core@<some_version> is correct
|
||||
|
||||
ARG dbt_core_ref=dbt-core@v1.2.0b1
|
||||
ARG dbt_postgres_ref=dbt-core@v1.2.0b1
|
||||
ARG dbt_redshift_ref=dbt-redshift@v1.0.0
|
||||
ARG dbt_bigquery_ref=dbt-bigquery@v1.0.0
|
||||
ARG dbt_snowflake_ref=dbt-snowflake@v1.0.0
|
||||
ARG dbt_spark_ref=dbt-spark@v1.0.0
|
||||
ARG dbt_core_ref=dbt-core@v1.2.0
|
||||
ARG dbt_postgres_ref=dbt-core@v1.2.0
|
||||
ARG dbt_redshift_ref=dbt-redshift@v1.2.0
|
||||
ARG dbt_bigquery_ref=dbt-bigquery@v1.2.0
|
||||
ARG dbt_snowflake_ref=dbt-snowflake@v1.2.0
|
||||
ARG dbt_spark_ref=dbt-spark@v1.2.0
|
||||
# special case args
|
||||
ARG dbt_spark_version=all
|
||||
ARG dbt_third_party
|
||||
|
||||
@@ -1 +1 @@
|
||||
version = "1.2.0b1"
|
||||
version = "1.2.0"
|
||||
|
||||
@@ -31,6 +31,7 @@ class PostgresCredentials(Credentials):
|
||||
sslkey: Optional[str] = None
|
||||
sslrootcert: Optional[str] = None
|
||||
application_name: Optional[str] = "dbt"
|
||||
retries: int = 1
|
||||
|
||||
_ALIASES = {"dbname": "database", "pass": "password"}
|
||||
|
||||
@@ -121,7 +122,7 @@ class PostgresConnectionManager(SQLConnectionManager):
|
||||
if credentials.application_name:
|
||||
kwargs["application_name"] = credentials.application_name
|
||||
|
||||
try:
|
||||
def connect():
|
||||
handle = psycopg2.connect(
|
||||
dbname=credentials.database,
|
||||
user=credentials.user,
|
||||
@@ -131,23 +132,26 @@ class PostgresConnectionManager(SQLConnectionManager):
|
||||
connect_timeout=credentials.connect_timeout,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
if credentials.role:
|
||||
handle.cursor().execute("set role {}".format(credentials.role))
|
||||
return handle
|
||||
|
||||
connection.handle = handle
|
||||
connection.state = "open"
|
||||
except psycopg2.Error as e:
|
||||
logger.debug(
|
||||
"Got an error when attempting to open a postgres " "connection: '{}'".format(e)
|
||||
)
|
||||
retryable_exceptions = [
|
||||
# OperationalError is subclassed by all psycopg2 Connection Exceptions and it's raised
|
||||
# by generic connection timeouts without an error code. This is a limitation of
|
||||
# psycopg2 which doesn't provide subclasses for errors without a SQLSTATE error code.
|
||||
# The limitation has been known for a while and there are no efforts to tackle it.
|
||||
# See: https://github.com/psycopg/psycopg2/issues/682
|
||||
psycopg2.errors.OperationalError,
|
||||
]
|
||||
|
||||
connection.handle = None
|
||||
connection.state = "fail"
|
||||
|
||||
raise dbt.exceptions.FailedToConnectException(str(e))
|
||||
|
||||
return connection
|
||||
return cls.retry_connection(
|
||||
connection,
|
||||
connect=connect,
|
||||
logger=logger,
|
||||
retry_limit=credentials.retries,
|
||||
retryable_exceptions=retryable_exceptions,
|
||||
)
|
||||
|
||||
def cancel(self, connection):
|
||||
connection_name = connection.name
|
||||
|
||||
@@ -41,7 +41,7 @@ def _dbt_psycopg2_name():
|
||||
|
||||
|
||||
package_name = "dbt-postgres"
|
||||
package_version = "1.2.0b1"
|
||||
package_version = "1.2.0"
|
||||
description = """The postgres adapter plugin for dbt (data build tool)"""
|
||||
|
||||
this_directory = os.path.abspath(os.path.dirname(__file__))
|
||||
|
||||
494
test/unit/test_adapter_connection_manager.py
Normal file
494
test/unit/test_adapter_connection_manager.py
Normal file
@@ -0,0 +1,494 @@
|
||||
import unittest
|
||||
from unittest import mock
|
||||
import sys
|
||||
|
||||
import dbt.exceptions
|
||||
|
||||
import psycopg2
|
||||
|
||||
from dbt.contracts.connection import Connection
|
||||
from dbt.adapters.base import BaseConnectionManager
|
||||
from dbt.adapters.postgres import PostgresCredentials, PostgresConnectionManager
|
||||
from dbt.events import AdapterLogger
|
||||
|
||||
|
||||
class BaseConnectionManagerTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.postgres_credentials = PostgresCredentials(
|
||||
host="localhost",
|
||||
user="test-user",
|
||||
port=1111,
|
||||
password="test-password",
|
||||
database="test-db",
|
||||
schema="test-schema",
|
||||
)
|
||||
self.logger = AdapterLogger("test")
|
||||
self.postgres_connection = Connection("postgres", None, self.postgres_credentials)
|
||||
|
||||
def test_retry_connection(self):
|
||||
"""Test a dummy handle is set on a connection on the first attempt.
|
||||
|
||||
This test uses a Connection populated with test PostgresCredentials values, and
|
||||
expects the Connection.handle attribute to be set to True and it's state to
|
||||
"open", after calling retry_connection.
|
||||
|
||||
Moreover, the attribute should be set in the first attempt as no exception would
|
||||
be raised for retrying. A mock connect function is used to simulate a real connection
|
||||
passing on the first attempt.
|
||||
"""
|
||||
conn = self.postgres_connection
|
||||
attempts = 0
|
||||
|
||||
def connect():
|
||||
nonlocal attempts
|
||||
attempts += 1
|
||||
return True
|
||||
|
||||
conn = BaseConnectionManager.retry_connection(
|
||||
conn,
|
||||
connect,
|
||||
self.logger,
|
||||
retryable_exceptions=[],
|
||||
)
|
||||
|
||||
assert conn.state == "open"
|
||||
assert conn.handle is True
|
||||
assert attempts == 1
|
||||
|
||||
def test_retry_connection_fails_unhandled(self):
|
||||
"""Test setting a handle fails upon raising a non-handled exception.
|
||||
|
||||
This test uses a Connection populated with test PostgresCredentials values, and
|
||||
expects a ValueError to be raised by a mock connect function. As a
|
||||
result:
|
||||
* The Connection state should be "fail" and the handle None.
|
||||
* The resulting attempt count should be 1 as we are not explicitly configured to handle a
|
||||
ValueError.
|
||||
* retry_connection should raise a FailedToConnectException with the Exception message.
|
||||
"""
|
||||
conn = self.postgres_connection
|
||||
attempts = 0
|
||||
|
||||
def connect():
|
||||
nonlocal attempts
|
||||
attempts += 1
|
||||
raise ValueError("Something went horribly wrong")
|
||||
|
||||
with self.assertRaisesRegex(
|
||||
dbt.exceptions.FailedToConnectException, "Something went horribly wrong"
|
||||
):
|
||||
|
||||
BaseConnectionManager.retry_connection(
|
||||
conn,
|
||||
connect,
|
||||
self.logger,
|
||||
retry_limit=1,
|
||||
retry_timeout=lambda attempt: 0,
|
||||
retryable_exceptions=(TypeError,),
|
||||
)
|
||||
|
||||
assert conn.state == "fail"
|
||||
assert conn.handle is None
|
||||
assert attempts == 1
|
||||
|
||||
def test_retry_connection_fails_handled(self):
|
||||
"""Test setting a handle fails upon raising a handled exception.
|
||||
|
||||
This test uses a Connection populated with test PostgresCredentials values, and
|
||||
expects a ValueError to be raised by a mock connect function.
|
||||
As a result:
|
||||
* The Connection state should be "fail" and the handle None.
|
||||
* The resulting attempt count should be 2 as we are configured to handle a ValueError.
|
||||
* retry_connection should raise a FailedToConnectException with the Exception message.
|
||||
"""
|
||||
conn = self.postgres_connection
|
||||
attempts = 0
|
||||
|
||||
def connect():
|
||||
nonlocal attempts
|
||||
attempts += 1
|
||||
raise ValueError("Something went horribly wrong")
|
||||
|
||||
with self.assertRaisesRegex(
|
||||
dbt.exceptions.FailedToConnectException, "Something went horribly wrong"
|
||||
):
|
||||
|
||||
BaseConnectionManager.retry_connection(
|
||||
conn,
|
||||
connect,
|
||||
self.logger,
|
||||
retry_timeout=0,
|
||||
retryable_exceptions=(ValueError,),
|
||||
retry_limit=1,
|
||||
)
|
||||
|
||||
assert conn.state == "fail"
|
||||
assert conn.handle is None
|
||||
|
||||
def test_retry_connection_passes_handled(self):
|
||||
"""Test setting a handle fails upon raising a handled exception.
|
||||
|
||||
This test uses a Connection populated with test PostgresCredentials values, and
|
||||
expects a ValueError to be raised by a mock connect function only the first
|
||||
time is called. Upon handling the exception once, connect should return.
|
||||
As a result:
|
||||
* The Connection state should be "open" and the handle True.
|
||||
* The resulting attempt count should be 2 as we are configured to handle a ValueError.
|
||||
"""
|
||||
conn = self.postgres_connection
|
||||
is_handled = False
|
||||
attempts = 0
|
||||
|
||||
def connect():
|
||||
nonlocal is_handled
|
||||
nonlocal attempts
|
||||
|
||||
attempts += 1
|
||||
|
||||
if is_handled:
|
||||
return True
|
||||
|
||||
is_handled = True
|
||||
raise ValueError("Something went horribly wrong")
|
||||
|
||||
conn = BaseConnectionManager.retry_connection(
|
||||
conn,
|
||||
connect,
|
||||
self.logger,
|
||||
retry_timeout=0,
|
||||
retryable_exceptions=(ValueError,),
|
||||
retry_limit=1,
|
||||
)
|
||||
|
||||
assert conn.state == "open"
|
||||
assert conn.handle is True
|
||||
assert is_handled is True
|
||||
assert attempts == 2
|
||||
|
||||
def test_retry_connection_attempts(self):
|
||||
"""Test setting a handle fails upon raising a handled exception multiple times.
|
||||
|
||||
This test uses a Connection populated with test PostgresCredentials values, and
|
||||
expects a ValueError to be raised by a mock connect function. As a result:
|
||||
* The Connection state should be "fail" and the handle None, as connect
|
||||
never returns.
|
||||
* The resulting attempt count should be 11 as we are configured to handle a ValueError.
|
||||
* retry_connection should raise a FailedToConnectException with the Exception message.
|
||||
"""
|
||||
conn = self.postgres_connection
|
||||
attempts = 0
|
||||
|
||||
def connect():
|
||||
nonlocal attempts
|
||||
attempts += 1
|
||||
|
||||
raise ValueError("Something went horribly wrong")
|
||||
|
||||
with self.assertRaisesRegex(
|
||||
dbt.exceptions.FailedToConnectException, "Something went horribly wrong"
|
||||
):
|
||||
BaseConnectionManager.retry_connection(
|
||||
conn,
|
||||
connect,
|
||||
self.logger,
|
||||
retry_timeout=0,
|
||||
retryable_exceptions=(ValueError,),
|
||||
retry_limit=10,
|
||||
)
|
||||
|
||||
assert conn.state == "fail"
|
||||
assert conn.handle is None
|
||||
assert attempts == 11
|
||||
|
||||
def test_retry_connection_fails_handling_all_exceptions(self):
|
||||
"""Test setting a handle fails after exhausting all attempts.
|
||||
|
||||
This test uses a Connection populated with test PostgresCredentials values, and
|
||||
expects a TypeError to be raised by a mock connect function. As a result:
|
||||
* The Connection state should be "fail" and the handle None, as connect
|
||||
never returns.
|
||||
* The resulting attempt count should be 11 as we are configured to handle all Exceptions.
|
||||
* retry_connection should raise a FailedToConnectException with the Exception message.
|
||||
"""
|
||||
conn = self.postgres_connection
|
||||
attempts = 0
|
||||
|
||||
def connect():
|
||||
nonlocal attempts
|
||||
attempts += 1
|
||||
|
||||
raise TypeError("An unhandled thing went horribly wrong")
|
||||
|
||||
with self.assertRaisesRegex(
|
||||
dbt.exceptions.FailedToConnectException, "An unhandled thing went horribly wrong"
|
||||
):
|
||||
BaseConnectionManager.retry_connection(
|
||||
conn,
|
||||
connect,
|
||||
self.logger,
|
||||
retry_timeout=0,
|
||||
retryable_exceptions=[Exception],
|
||||
retry_limit=15,
|
||||
)
|
||||
|
||||
assert conn.state == "fail"
|
||||
assert conn.handle is None
|
||||
assert attempts == 16
|
||||
|
||||
def test_retry_connection_passes_multiple_handled(self):
|
||||
"""Test setting a handle passes upon handling multiple exceptions.
|
||||
|
||||
This test uses a Connection populated with test PostgresCredentials values, and
|
||||
expects a mock connect to raise a ValueError in the first invocation and a
|
||||
TypeError in the second invocation. As a result:
|
||||
* The Connection state should be "open" and the handle True, as connect
|
||||
returns after both exceptions have been handled.
|
||||
* The resulting attempt count should be 3.
|
||||
"""
|
||||
conn = self.postgres_connection
|
||||
is_value_err_handled = False
|
||||
is_type_err_handled = False
|
||||
attempts = 0
|
||||
|
||||
def connect():
|
||||
nonlocal is_value_err_handled
|
||||
nonlocal is_type_err_handled
|
||||
nonlocal attempts
|
||||
|
||||
attempts += 1
|
||||
|
||||
if is_value_err_handled and is_type_err_handled:
|
||||
return True
|
||||
elif is_type_err_handled:
|
||||
is_value_err_handled = True
|
||||
raise ValueError("Something went horribly wrong")
|
||||
else:
|
||||
is_type_err_handled = True
|
||||
raise TypeError("An unhandled thing went horribly wrong")
|
||||
|
||||
conn = BaseConnectionManager.retry_connection(
|
||||
conn,
|
||||
connect,
|
||||
self.logger,
|
||||
retry_timeout=0,
|
||||
retryable_exceptions=(ValueError, TypeError),
|
||||
retry_limit=2,
|
||||
)
|
||||
|
||||
assert conn.state == "open"
|
||||
assert conn.handle is True
|
||||
assert is_type_err_handled is True
|
||||
assert is_value_err_handled is True
|
||||
assert attempts == 3
|
||||
|
||||
def test_retry_connection_passes_none_excluded(self):
|
||||
"""Test setting a handle passes upon handling multiple exceptions.
|
||||
|
||||
This test uses a Connection populated with test PostgresCredentials values, and
|
||||
expects a mock connect to raise a ValueError in the first invocation and a
|
||||
TypeError in the second invocation. As a result:
|
||||
* The Connection state should be "open" and the handle True, as connect
|
||||
returns after both exceptions have been handled.
|
||||
* The resulting attempt count should be 3.
|
||||
"""
|
||||
conn = self.postgres_connection
|
||||
is_value_err_handled = False
|
||||
is_type_err_handled = False
|
||||
attempts = 0
|
||||
|
||||
def connect():
|
||||
nonlocal is_value_err_handled
|
||||
nonlocal is_type_err_handled
|
||||
nonlocal attempts
|
||||
|
||||
attempts += 1
|
||||
|
||||
if is_value_err_handled and is_type_err_handled:
|
||||
return True
|
||||
elif is_type_err_handled:
|
||||
is_value_err_handled = True
|
||||
raise ValueError("Something went horribly wrong")
|
||||
else:
|
||||
is_type_err_handled = True
|
||||
raise TypeError("An unhandled thing went horribly wrong")
|
||||
|
||||
conn = BaseConnectionManager.retry_connection(
|
||||
conn,
|
||||
connect,
|
||||
self.logger,
|
||||
retry_timeout=0,
|
||||
retryable_exceptions=(ValueError, TypeError),
|
||||
retry_limit=2,
|
||||
)
|
||||
|
||||
assert conn.state == "open"
|
||||
assert conn.handle is True
|
||||
assert is_type_err_handled is True
|
||||
assert is_value_err_handled is True
|
||||
assert attempts == 3
|
||||
|
||||
def test_retry_connection_retry_limit(self):
|
||||
"""Test retry_connection raises an exception with a negative retry limit."""
|
||||
conn = self.postgres_connection
|
||||
attempts = 0
|
||||
|
||||
def connect():
|
||||
nonlocal attempts
|
||||
attempts += 1
|
||||
return True
|
||||
|
||||
with self.assertRaisesRegex(
|
||||
dbt.exceptions.FailedToConnectException, "retry_limit cannot be negative"
|
||||
):
|
||||
BaseConnectionManager.retry_connection(
|
||||
conn,
|
||||
connect,
|
||||
self.logger,
|
||||
retry_timeout=0,
|
||||
retryable_exceptions=(ValueError,),
|
||||
retry_limit=-2,
|
||||
)
|
||||
|
||||
assert conn.state == "fail"
|
||||
assert conn.handle is None
|
||||
assert attempts == 0
|
||||
|
||||
def test_retry_connection_retry_timeout(self):
|
||||
"""Test retry_connection raises an exception with a negative timeout."""
|
||||
conn = self.postgres_connection
|
||||
attempts = 0
|
||||
|
||||
def connect():
|
||||
nonlocal attempts
|
||||
attempts += 1
|
||||
return True
|
||||
|
||||
for retry_timeout in [-10, -2.5, lambda _: -100, lambda _: -10.1]:
|
||||
with self.assertRaisesRegex(
|
||||
dbt.exceptions.FailedToConnectException,
|
||||
"retry_timeout cannot be negative or return a negative time",
|
||||
):
|
||||
BaseConnectionManager.retry_connection(
|
||||
conn,
|
||||
connect,
|
||||
self.logger,
|
||||
retry_timeout=-10,
|
||||
retryable_exceptions=(ValueError,),
|
||||
retry_limit=2,
|
||||
)
|
||||
|
||||
assert conn.state == "init"
|
||||
assert conn.handle is None
|
||||
assert attempts == 0
|
||||
|
||||
def test_retry_connection_exceeds_recursion_limit(self):
|
||||
"""Test retry_connection raises an exception with retries that exceed recursion limit."""
|
||||
conn = self.postgres_connection
|
||||
attempts = 0
|
||||
|
||||
def connect():
|
||||
nonlocal attempts
|
||||
attempts += 1
|
||||
return True
|
||||
|
||||
with self.assertRaisesRegex(
|
||||
dbt.exceptions.FailedToConnectException,
|
||||
"retry_limit cannot be negative",
|
||||
):
|
||||
BaseConnectionManager.retry_connection(
|
||||
conn,
|
||||
connect,
|
||||
self.logger,
|
||||
retry_timeout=2,
|
||||
retryable_exceptions=(ValueError,),
|
||||
retry_limit=sys.getrecursionlimit() + 1,
|
||||
)
|
||||
|
||||
assert conn.state == "fail"
|
||||
assert conn.handle is None
|
||||
assert attempts == 0
|
||||
|
||||
def test_retry_connection_with_exponential_backoff_timeout(self):
|
||||
"""Test retry_connection with an exponential backoff timeout.
|
||||
|
||||
We assert the provided exponential backoff function gets passed the right attempt number
|
||||
and produces the expected timeouts.
|
||||
"""
|
||||
conn = self.postgres_connection
|
||||
attempts = 0
|
||||
timeouts = []
|
||||
|
||||
def connect():
|
||||
nonlocal attempts
|
||||
attempts += 1
|
||||
|
||||
if attempts < 12:
|
||||
raise ValueError("Keep trying!")
|
||||
return True
|
||||
|
||||
def exp_backoff(n):
|
||||
nonlocal timeouts
|
||||
computed = 2**n
|
||||
# We store the computed values to ensure they match the expected backoff...
|
||||
timeouts.append((n, computed))
|
||||
# but we return 0 as we don't want the test to go on forever.
|
||||
return 0
|
||||
|
||||
conn = BaseConnectionManager.retry_connection(
|
||||
conn,
|
||||
connect,
|
||||
self.logger,
|
||||
retry_timeout=exp_backoff,
|
||||
retryable_exceptions=(ValueError,),
|
||||
retry_limit=12,
|
||||
)
|
||||
|
||||
assert conn.state == "open"
|
||||
assert conn.handle is True
|
||||
assert attempts == 12
|
||||
assert timeouts == [(n, 2**n) for n in range(12)]
|
||||
|
||||
|
||||
class PostgresConnectionManagerTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.credentials = PostgresCredentials(
|
||||
host="localhost",
|
||||
user="test-user",
|
||||
port=1111,
|
||||
password="test-password",
|
||||
database="test-db",
|
||||
schema="test-schema",
|
||||
retries=2,
|
||||
)
|
||||
self.connection = Connection("postgres", None, self.credentials)
|
||||
|
||||
def test_open(self):
|
||||
"""Test opening a Postgres Connection with failures in the first 3 attempts.
|
||||
|
||||
This test uses a Connection populated with test PostgresCredentials values, and
|
||||
expects a mock connect to raise a psycopg2.errors.ConnectionFailuer
|
||||
in the first 3 invocations, after which the mock should return True. As a result:
|
||||
* The Connection state should be "open" and the handle True, as connect
|
||||
returns in the 4th attempt.
|
||||
* The resulting attempt count should be 4.
|
||||
"""
|
||||
conn = self.connection
|
||||
attempt = 0
|
||||
|
||||
def connect(*args, **kwargs):
|
||||
nonlocal attempt
|
||||
attempt += 1
|
||||
|
||||
if attempt <= 2:
|
||||
raise psycopg2.errors.ConnectionFailure("Connection has failed")
|
||||
|
||||
return True
|
||||
|
||||
with mock.patch("psycopg2.connect", wraps=connect) as mock_connect:
|
||||
PostgresConnectionManager.open(conn)
|
||||
|
||||
assert mock_connect.call_count == 3
|
||||
|
||||
assert attempt == 3
|
||||
assert conn.state == "open"
|
||||
assert conn.handle is True
|
||||
@@ -190,9 +190,9 @@ REQUIRED_BASE_KEYS = frozenset(
|
||||
"fromyaml",
|
||||
"toyaml",
|
||||
"set",
|
||||
"try_set",
|
||||
"set_strict",
|
||||
"zip",
|
||||
"try_zip",
|
||||
"zip_strict",
|
||||
"log",
|
||||
"run_started_at",
|
||||
"invocation_id",
|
||||
|
||||
@@ -1 +1 @@
|
||||
version = "1.2.0b1"
|
||||
version = "1.2.0"
|
||||
|
||||
@@ -20,7 +20,7 @@ except ImportError:
|
||||
|
||||
|
||||
package_name = "dbt-tests-adapter"
|
||||
package_version = "1.2.0b1"
|
||||
package_version = "1.2.0"
|
||||
description = """The dbt adapter tests for adapter plugins"""
|
||||
|
||||
this_directory = os.path.abspath(os.path.dirname(__file__))
|
||||
|
||||
@@ -7,8 +7,8 @@ macros__validate_set_sql = """
|
||||
{% macro validate_set() %}
|
||||
{% set set_result = set([1, 2, 2, 3, 'foo', False]) %}
|
||||
{{ log("set_result: " ~ set_result) }}
|
||||
{% set try_set_result = try_set([1, 2, 2, 3, 'foo', False]) %}
|
||||
{{ log("try_set_result: " ~ try_set_result) }}
|
||||
{% set set_strict_result = set_strict([1, 2, 2, 3, 'foo', False]) %}
|
||||
{{ log("set_strict_result: " ~ set_strict_result) }}
|
||||
{% endmacro %}
|
||||
"""
|
||||
|
||||
@@ -18,17 +18,17 @@ macros__validate_zip_sql = """
|
||||
{% set list_b = ['foo', 'bar'] %}
|
||||
{% set zip_result = zip(list_a, list_b) | list %}
|
||||
{{ log("zip_result: " ~ zip_result) }}
|
||||
{% set try_zip_result = try_zip(list_a, list_b) | list %}
|
||||
{{ log("try_zip_result: " ~ try_zip_result) }}
|
||||
{% set zip_strict_result = zip_strict(list_a, list_b) | list %}
|
||||
{{ log("zip_strict_result: " ~ zip_strict_result) }}
|
||||
{% endmacro %}
|
||||
"""
|
||||
|
||||
models__set_exception_sql = """
|
||||
{% set try_set_result = try_set(1) %}
|
||||
{% set set_strict_result = set_strict(1) %}
|
||||
"""
|
||||
|
||||
models__zip_exception_sql = """
|
||||
{% set try_set_result = try_zip(1) %}
|
||||
{% set zip_strict_result = zip_strict(1) %}
|
||||
"""
|
||||
|
||||
|
||||
@@ -46,18 +46,18 @@ class TestContextBuiltins:
|
||||
# The order of the set isn't guaranteed so we can't check for the actual set in the logs
|
||||
assert "set_result: " in log_output
|
||||
assert "False" in log_output
|
||||
assert "try_set_result: " in log_output
|
||||
assert "set_strict_result: " in log_output
|
||||
|
||||
def test_builtin_zip_function(self, project):
|
||||
_, log_output = run_dbt_and_capture(["--debug", "run-operation", "validate_zip"])
|
||||
|
||||
expected_zip = [(1, "foo"), (2, "bar")]
|
||||
assert f"zip_result: {expected_zip}" in log_output
|
||||
assert f"try_zip_result: {expected_zip}" in log_output
|
||||
assert f"zip_strict_result: {expected_zip}" in log_output
|
||||
|
||||
|
||||
class TestContextBuiltinExceptions:
|
||||
# Assert compilation errors are raised with try_ equivalents
|
||||
# Assert compilation errors are raised with _strict equivalents
|
||||
def test_builtin_function_exception(self, project):
|
||||
write_file(models__set_exception_sql, project.project_root, "models", "raise.sql")
|
||||
with pytest.raises(CompilationException):
|
||||
|
||||
Reference in New Issue
Block a user