forked from repo-mirrors/dbt-core
Compare commits
49 Commits
arky/exp-i
...
test-sqlpa
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e603d9e2fa | ||
|
|
7d1c7518f8 | ||
|
|
30f8c6269a | ||
|
|
65fcbe7b79 | ||
|
|
f745e7c823 | ||
|
|
14f966c24f | ||
|
|
d0a6ea96e4 | ||
|
|
f3f1244eb0 | ||
|
|
0c067350da | ||
|
|
57a6963913 | ||
|
|
d2c9ee46a1 | ||
|
|
89fa60d86a | ||
|
|
e39505fec6 | ||
|
|
c82795a99f | ||
|
|
f0f4af5c8d | ||
|
|
3fe1502c15 | ||
|
|
cc2f259108 | ||
|
|
51d216186d | ||
|
|
bdf7c1ff82 | ||
|
|
f23dbb6f6b | ||
|
|
ec17ecec86 | ||
|
|
a3d326404d | ||
|
|
c785f320ea | ||
|
|
1fe0c32890 | ||
|
|
11752edef8 | ||
|
|
16cb498b56 | ||
|
|
03f6655f1a | ||
|
|
8dc3d0a531 | ||
|
|
c172687736 | ||
|
|
8a31eab181 | ||
|
|
4d0ee2fc47 | ||
|
|
ddb2f0f71d | ||
|
|
2bb2e73df9 | ||
|
|
790fecab92 | ||
|
|
f6926c0ed6 | ||
|
|
52be5ffaa6 | ||
|
|
31bfff81d9 | ||
|
|
3fbfa6e6f3 | ||
|
|
165035692f | ||
|
|
80f76e9e6e | ||
|
|
a2e7249c44 | ||
|
|
9700ff1866 | ||
|
|
e61a39f27f | ||
|
|
d934e713db | ||
|
|
ef9bb925d3 | ||
|
|
f73359b87c | ||
|
|
b4706c4dec | ||
|
|
b46d35c13f | ||
|
|
eba90863ed |
@@ -1,13 +1,15 @@
|
||||
[bumpversion]
|
||||
current_version = 1.4.0b1
|
||||
parse = (?P<major>\d+)
|
||||
\.(?P<minor>\d+)
|
||||
\.(?P<patch>\d+)
|
||||
((?P<prekind>a|b|rc)
|
||||
(?P<pre>\d+) # pre-release version num
|
||||
current_version = 1.4.5
|
||||
parse = (?P<major>[\d]+) # major version number
|
||||
\.(?P<minor>[\d]+) # minor version number
|
||||
\.(?P<patch>[\d]+) # patch version number
|
||||
(((?P<prekind>a|b|rc) # optional pre-release type
|
||||
?(?P<num>[\d]+?)) # optional pre-release version number
|
||||
\.?(?P<nightly>[a-z0-9]+\+[a-z]+)? # optional nightly release indicator
|
||||
)?
|
||||
serialize =
|
||||
{major}.{minor}.{patch}{prekind}{pre}
|
||||
{major}.{minor}.{patch}{prekind}{num}.{nightly}
|
||||
{major}.{minor}.{patch}{prekind}{num}
|
||||
{major}.{minor}.{patch}
|
||||
commit = False
|
||||
tag = False
|
||||
@@ -21,9 +23,11 @@ values =
|
||||
rc
|
||||
final
|
||||
|
||||
[bumpversion:part:pre]
|
||||
[bumpversion:part:num]
|
||||
first_value = 1
|
||||
|
||||
[bumpversion:part:nightly]
|
||||
|
||||
[bumpversion:file:core/setup.py]
|
||||
|
||||
[bumpversion:file:core/dbt/version.py]
|
||||
|
||||
@@ -1,9 +1,14 @@
|
||||
## dbt-core 1.4.0-b1 - December 15, 2022
|
||||
## dbt-core 1.4.0 - January 25, 2023
|
||||
|
||||
### Breaking Changes
|
||||
|
||||
- Cleaned up exceptions to directly raise in code. Also updated the existing exception to meet PEP guidelines.Removed use of all exception functions in the code base and marked them all as deprecated to be removed next minor release. ([#6339](https://github.com/dbt-labs/dbt-core/issues/6339), [#6393](https://github.com/dbt-labs/dbt-core/issues/6393), [#6460](https://github.com/dbt-labs/dbt-core/issues/6460))
|
||||
|
||||
### Features
|
||||
|
||||
- Added favor-state flag to optionally favor state nodes even if unselected node exists ([#2968](https://github.com/dbt-labs/dbt-core/issues/2968))
|
||||
- Added favor-state flag to optionally favor state nodes even if unselected node exists ([#5016](https://github.com/dbt-labs/dbt-core/issues/5016))
|
||||
- Update structured logging. Convert to using protobuf messages. Ensure events are enriched with node_info. ([#5610](https://github.com/dbt-labs/dbt-core/issues/5610))
|
||||
- incremental predicates ([#5680](https://github.com/dbt-labs/dbt-core/issues/5680))
|
||||
- Friendlier error messages when packages.yml is malformed ([#5486](https://github.com/dbt-labs/dbt-core/issues/5486))
|
||||
- Allow partitions in external tables to be supplied as a list ([#5929](https://github.com/dbt-labs/dbt-core/issues/5929))
|
||||
- extend -f flag shorthand for seed command ([#5990](https://github.com/dbt-labs/dbt-core/issues/5990))
|
||||
@@ -12,7 +17,10 @@
|
||||
- Added an md5 function to the base context ([#6246](https://github.com/dbt-labs/dbt-core/issues/6246))
|
||||
- Exposures support metrics in lineage ([#6057](https://github.com/dbt-labs/dbt-core/issues/6057))
|
||||
- Add support for Python 3.11 ([#6147](https://github.com/dbt-labs/dbt-core/issues/6147))
|
||||
- incremental predicates ([#5680](https://github.com/dbt-labs/dbt-core/issues/5680))
|
||||
- Making timestamp optional for metrics ([#6398](https://github.com/dbt-labs/dbt-core/issues/6398))
|
||||
- The meta configuration field is now included in the node_info property of structured logs. ([#6216](https://github.com/dbt-labs/dbt-core/issues/6216))
|
||||
- Adds buildable selection mode ([#6365](https://github.com/dbt-labs/dbt-core/issues/6365))
|
||||
- --warn-error-options: Treat warnings as errors for specific events, based on user configuration ([#6165](https://github.com/dbt-labs/dbt-core/issues/6165))
|
||||
|
||||
### Fixes
|
||||
|
||||
@@ -21,11 +29,25 @@
|
||||
- check length of args of python model function before accessing it ([#6041](https://github.com/dbt-labs/dbt-core/issues/6041))
|
||||
- Add functors to ensure event types with str-type attributes are initialized to spec, even when provided non-str type params. ([#5436](https://github.com/dbt-labs/dbt-core/issues/5436))
|
||||
- Allow hooks to fail without halting execution flow ([#5625](https://github.com/dbt-labs/dbt-core/issues/5625))
|
||||
- fix missing f-strings, convert old .format() messages to f-strings for consistency ([#6241](https://github.com/dbt-labs/dbt-core/issues/6241))
|
||||
- Clarify Error Message for how many models are allowed in a Python file ([#6245](https://github.com/dbt-labs/dbt-core/issues/6245))
|
||||
- Fix typo in util.py ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904))
|
||||
- After this, will be possible to use default values for dbt.config.get ([#6309](https://github.com/dbt-labs/dbt-core/issues/6309))
|
||||
- Use full path for writing manifest ([#6055](https://github.com/dbt-labs/dbt-core/issues/6055))
|
||||
- [CT-1284] Change Python model default materialization to table ([#6345](https://github.com/dbt-labs/dbt-core/issues/6345))
|
||||
- add pre-commit install to make dev script in Makefile ([#6269](https://github.com/dbt-labs/dbt-core/issues/6269))
|
||||
- Late-rendering for `pre_` and `post_hook`s in `dbt_project.yml` ([#6411](https://github.com/dbt-labs/dbt-core/issues/6411))
|
||||
- [CT-1284] Change Python model default materialization to table ([#5989](https://github.com/dbt-labs/dbt-core/issues/5989))
|
||||
- [CT-1591] Don't parse empty Python files ([#6345](https://github.com/dbt-labs/dbt-core/issues/6345))
|
||||
- Repair a regression which prevented basic logging before the logging subsystem is completely configured. ([#6434](https://github.com/dbt-labs/dbt-core/issues/6434))
|
||||
- fix docs generate --defer by adding defer_to_manifest to before_run ([#6488](https://github.com/dbt-labs/dbt-core/issues/6488))
|
||||
- Bug when partial parsing with an empty schema file ([#4850](https://github.com/dbt-labs/dbt-core/issues/4850))
|
||||
- Fix DBT_FAVOR_STATE env var ([#5859](https://github.com/dbt-labs/dbt-core/issues/5859))
|
||||
- Restore historical behavior of certain disabled test messages, so that they are at the less obtrusive debug level, rather than the warning level. ([#6501](https://github.com/dbt-labs/dbt-core/issues/6501))
|
||||
- Bump mashumuro version to get regression fix and add unit test to verify that fix. ([#6428](https://github.com/dbt-labs/dbt-core/issues/6428))
|
||||
- Call update_event_status earlier for node results. Rename event 'HookFinished' -> FinishedRunningStats ([#6571](https://github.com/dbt-labs/dbt-core/issues/6571))
|
||||
- Provide backward compatibility for `get_merge_sql` arguments ([#6625](https://github.com/dbt-labs/dbt-core/issues/6625))
|
||||
- Fix behavior of --favor-state with --defer ([#6617](https://github.com/dbt-labs/dbt-core/issues/6617))
|
||||
- Include adapter_response in NodeFinished run_result log event ([#6703](https://github.com/dbt-labs/dbt-core/issues/6703))
|
||||
|
||||
### Docs
|
||||
|
||||
@@ -34,6 +56,7 @@
|
||||
- ([dbt-docs/#5880](https://github.com/dbt-labs/dbt-docs/issues/5880))
|
||||
- Fix rendering of sample code for metrics ([dbt-docs/#323](https://github.com/dbt-labs/dbt-docs/issues/323))
|
||||
- Alphabetize `core/dbt/README.md` ([dbt-docs/#6368](https://github.com/dbt-labs/dbt-docs/issues/6368))
|
||||
- Updated minor typos encountered when skipping profile setup ([dbt-docs/#6529](https://github.com/dbt-labs/dbt-docs/issues/6529))
|
||||
|
||||
### Under the Hood
|
||||
|
||||
@@ -57,6 +80,14 @@
|
||||
- Fix intermittent database connection failure in Windows CI test ([#6394](https://github.com/dbt-labs/dbt-core/issues/6394))
|
||||
- Refactor and clean up manifest nodes ([#6426](https://github.com/dbt-labs/dbt-core/issues/6426))
|
||||
- Restore important legacy logging behaviors, following refactor which removed them ([#6437](https://github.com/dbt-labs/dbt-core/issues/6437))
|
||||
- Treat dense text blobs as binary for `git grep` ([#6294](https://github.com/dbt-labs/dbt-core/issues/6294))
|
||||
- Prune partial parsing logging events ([#6313](https://github.com/dbt-labs/dbt-core/issues/6313))
|
||||
- Updating the deprecation warning in the metric attributes renamed event ([#6507](https://github.com/dbt-labs/dbt-core/issues/6507))
|
||||
- [CT-1693] Port severity test to Pytest ([#6466](https://github.com/dbt-labs/dbt-core/issues/6466))
|
||||
- [CT-1694] Deprecate event tracking tests ([#6467](https://github.com/dbt-labs/dbt-core/issues/6467))
|
||||
- Reorganize structured logging events to have two top keys ([#6311](https://github.com/dbt-labs/dbt-core/issues/6311))
|
||||
- Combine some logging events ([#1716](https://github.com/dbt-labs/dbt-core/issues/1716), [#1717](https://github.com/dbt-labs/dbt-core/issues/1717), [#1719](https://github.com/dbt-labs/dbt-core/issues/1719))
|
||||
- Check length of escaped strings in the adapter test ([#6566](https://github.com/dbt-labs/dbt-core/issues/6566))
|
||||
|
||||
### Dependencies
|
||||
|
||||
@@ -64,25 +95,32 @@
|
||||
- Bump black from 22.8.0 to 22.10.0 ([#6019](https://github.com/dbt-labs/dbt-core/pull/6019))
|
||||
- Bump mashumaro[msgpack] from 3.0.4 to 3.1.1 in /core ([#6108](https://github.com/dbt-labs/dbt-core/pull/6108))
|
||||
- Update colorama requirement from <0.4.6,>=0.3.9 to >=0.3.9,<0.4.7 in /core ([#6144](https://github.com/dbt-labs/dbt-core/pull/6144))
|
||||
- Bump mashumaro[msgpack] from 3.1.1 to 3.2 in /core ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904))
|
||||
- Bump mashumaro[msgpack] from 3.1.1 to 3.2 in /core ([#6375](https://github.com/dbt-labs/dbt-core/pull/6375))
|
||||
- Update agate requirement from <1.6.4,>=1.6 to >=1.6,<1.7.1 in /core ([#6506](https://github.com/dbt-labs/dbt-core/pull/6506))
|
||||
|
||||
### Contributors
|
||||
- [@NiallRees](https://github.com/NiallRees) ([#5859](https://github.com/dbt-labs/dbt-core/issues/5859))
|
||||
- [@agpapa](https://github.com/agpapa) ([#6365](https://github.com/dbt-labs/dbt-core/issues/6365))
|
||||
- [@andy-clapson](https://github.com/andy-clapson) ([dbt-docs/#5791](https://github.com/dbt-labs/dbt-docs/issues/5791))
|
||||
- [@callum-mcdata](https://github.com/callum-mcdata) ([#6398](https://github.com/dbt-labs/dbt-core/issues/6398), [#6507](https://github.com/dbt-labs/dbt-core/issues/6507))
|
||||
- [@chamini2](https://github.com/chamini2) ([#6041](https://github.com/dbt-labs/dbt-core/issues/6041))
|
||||
- [@daniel-murray](https://github.com/daniel-murray) ([#2968](https://github.com/dbt-labs/dbt-core/issues/2968))
|
||||
- [@dave-connors-3](https://github.com/dave-connors-3) ([#5990](https://github.com/dbt-labs/dbt-core/issues/5990))
|
||||
- [@dbeatty10](https://github.com/dbeatty10) ([dbt-docs/#6368](https://github.com/dbt-labs/dbt-docs/issues/6368), [#6394](https://github.com/dbt-labs/dbt-core/issues/6394))
|
||||
- [@daniel-murray](https://github.com/daniel-murray) ([#5016](https://github.com/dbt-labs/dbt-core/issues/5016))
|
||||
- [@dave-connors-3](https://github.com/dave-connors-3) ([#5680](https://github.com/dbt-labs/dbt-core/issues/5680), [#5990](https://github.com/dbt-labs/dbt-core/issues/5990), [#6625](https://github.com/dbt-labs/dbt-core/issues/6625))
|
||||
- [@dbeatty10](https://github.com/dbeatty10) ([#6411](https://github.com/dbt-labs/dbt-core/issues/6411), [dbt-docs/#6368](https://github.com/dbt-labs/dbt-docs/issues/6368), [#6394](https://github.com/dbt-labs/dbt-core/issues/6394), [#6294](https://github.com/dbt-labs/dbt-core/issues/6294), [#6566](https://github.com/dbt-labs/dbt-core/issues/6566))
|
||||
- [@devmessias](https://github.com/devmessias) ([#6309](https://github.com/dbt-labs/dbt-core/issues/6309))
|
||||
- [@eltociear](https://github.com/eltociear) ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904))
|
||||
- [@eve-johns](https://github.com/eve-johns) ([#6068](https://github.com/dbt-labs/dbt-core/issues/6068))
|
||||
- [@haritamar](https://github.com/haritamar) ([#6246](https://github.com/dbt-labs/dbt-core/issues/6246))
|
||||
- [@jared-rimmer](https://github.com/jared-rimmer) ([#5486](https://github.com/dbt-labs/dbt-core/issues/5486))
|
||||
- [@josephberni](https://github.com/josephberni) ([#2968](https://github.com/dbt-labs/dbt-core/issues/2968))
|
||||
- [@josephberni](https://github.com/josephberni) ([#5016](https://github.com/dbt-labs/dbt-core/issues/5016))
|
||||
- [@joshuataylor](https://github.com/joshuataylor) ([#6147](https://github.com/dbt-labs/dbt-core/issues/6147))
|
||||
- [@justbldwn](https://github.com/justbldwn) ([#6245](https://github.com/dbt-labs/dbt-core/issues/6245))
|
||||
- [@justbldwn](https://github.com/justbldwn) ([#6241](https://github.com/dbt-labs/dbt-core/issues/6241), [#6245](https://github.com/dbt-labs/dbt-core/issues/6245), [#6269](https://github.com/dbt-labs/dbt-core/issues/6269))
|
||||
- [@luke-bassett](https://github.com/luke-bassett) ([#1350](https://github.com/dbt-labs/dbt-core/issues/1350))
|
||||
- [@max-sixty](https://github.com/max-sixty) ([#5946](https://github.com/dbt-labs/dbt-core/issues/5946), [#5983](https://github.com/dbt-labs/dbt-core/issues/5983), [#6028](https://github.com/dbt-labs/dbt-core/issues/6028))
|
||||
- [@mivanicova](https://github.com/mivanicova) ([#6488](https://github.com/dbt-labs/dbt-core/issues/6488))
|
||||
- [@nshuman1](https://github.com/nshuman1) ([dbt-docs/#6529](https://github.com/dbt-labs/dbt-docs/issues/6529))
|
||||
- [@paulbenschmidt](https://github.com/paulbenschmidt) ([dbt-docs/#5880](https://github.com/dbt-labs/dbt-docs/issues/5880))
|
||||
- [@pgoslatara](https://github.com/pgoslatara) ([#5929](https://github.com/dbt-labs/dbt-core/issues/5929))
|
||||
- [@racheldaniel](https://github.com/racheldaniel) ([#6201](https://github.com/dbt-labs/dbt-core/issues/6201))
|
||||
- [@timle2](https://github.com/timle2) ([#4205](https://github.com/dbt-labs/dbt-core/issues/4205))
|
||||
- [@dave-connors-3](https://github.com/dave-connors-3) ([#5680](https://github.com/dbt-labs/dbt-core/issues/5680))
|
||||
- [@tmastny](https://github.com/tmastny) ([#6216](https://github.com/dbt-labs/dbt-core/issues/6216))
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: "Dependencies"
|
||||
body: "Update pathspec requirement from ~=0.9.0 to >=0.9,<0.11 in /core"
|
||||
time: 2022-09-23T00:06:46.00000Z
|
||||
custom:
|
||||
Author: dependabot[bot]
|
||||
PR: "5917"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: "Dependencies"
|
||||
body: "Bump black from 22.8.0 to 22.10.0"
|
||||
time: 2022-10-07T00:08:48.00000Z
|
||||
custom:
|
||||
Author: dependabot[bot]
|
||||
PR: "6019"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: "Dependencies"
|
||||
body: "Bump mashumaro[msgpack] from 3.0.4 to 3.1.1 in /core"
|
||||
time: 2022-10-20T00:07:53.00000Z
|
||||
custom:
|
||||
Author: dependabot[bot]
|
||||
PR: "6108"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: "Dependencies"
|
||||
body: "Update colorama requirement from <0.4.6,>=0.3.9 to >=0.3.9,<0.4.7 in /core"
|
||||
time: 2022-10-26T00:09:10.00000Z
|
||||
custom:
|
||||
Author: dependabot[bot]
|
||||
PR: "6144"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: "Dependencies"
|
||||
body: "Bump mashumaro[msgpack] from 3.1.1 to 3.2 in /core"
|
||||
time: 2022-12-05T00:21:18.00000Z
|
||||
custom:
|
||||
Author: dependabot[bot]
|
||||
Issue: 4904
|
||||
PR: 6375
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Docs
|
||||
body: minor doc correction
|
||||
time: 2022-09-08T15:41:57.689162-04:00
|
||||
custom:
|
||||
Author: andy-clapson
|
||||
Issue: "5791"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Docs
|
||||
body: Generate API docs for new CLI interface
|
||||
time: 2022-10-07T09:06:56.446078-05:00
|
||||
custom:
|
||||
Author: stu-k
|
||||
Issue: "5528"
|
||||
@@ -1,5 +0,0 @@
|
||||
kind: Docs
|
||||
time: 2022-10-17T17:14:11.715348-05:00
|
||||
custom:
|
||||
Author: paulbenschmidt
|
||||
Issue: "5880"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Docs
|
||||
body: Fix rendering of sample code for metrics
|
||||
time: 2022-11-16T15:57:43.204201+01:00
|
||||
custom:
|
||||
Author: jtcohen6
|
||||
Issue: "323"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Docs
|
||||
body: Alphabetize `core/dbt/README.md`
|
||||
time: 2022-12-02T15:05:23.695333-07:00
|
||||
custom:
|
||||
Author: dbeatty10
|
||||
Issue: "6368"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Added favor-state flag to optionally favor state nodes even if unselected node
|
||||
exists
|
||||
time: 2022-04-08T16:54:59.696564+01:00
|
||||
custom:
|
||||
Author: daniel-murray josephberni
|
||||
Issue: "5016"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Features
|
||||
body: Update structured logging. Convert to using protobuf messages. Ensure events are enriched with node_info.
|
||||
time: 2022-08-17T15:48:57.225267-04:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "5610"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: incremental predicates
|
||||
time: 2022-08-23T08:57:27.640804-05:00
|
||||
custom:
|
||||
Author: dave-connors-3
|
||||
Issue: "5680"
|
||||
PR: "5702"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Features
|
||||
body: Friendlier error messages when packages.yml is malformed
|
||||
time: 2022-09-12T12:59:35.121188+01:00
|
||||
custom:
|
||||
Author: jared-rimmer
|
||||
Issue: "5486"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Features
|
||||
body: Allow partitions in external tables to be supplied as a list
|
||||
time: 2022-09-25T21:16:51.051239654+02:00
|
||||
custom:
|
||||
Author: pgoslatara
|
||||
Issue: "5929"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Features
|
||||
body: extend -f flag shorthand for seed command
|
||||
time: 2022-10-03T11:07:05.381632-05:00
|
||||
custom:
|
||||
Author: dave-connors-3
|
||||
Issue: "5990"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: This pulls the profile name from args when constructing a RuntimeConfig in lib.py,
|
||||
enabling the dbt-server to override the value that's in the dbt_project.yml
|
||||
time: 2022-11-02T15:00:03.000805-05:00
|
||||
custom:
|
||||
Author: racheldaniel
|
||||
Issue: "6201"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Features
|
||||
body: Adding tarball install method for packages. Allowing package tarball to be specified
|
||||
via url in the packages.yaml.
|
||||
time: 2022-11-07T10:50:18.464545-05:00
|
||||
custom:
|
||||
Author: timle2
|
||||
Issue: "4205"
|
||||
PR: "4689"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Features
|
||||
body: Added an md5 function to the base context
|
||||
time: 2022-11-14T18:52:07.788593+02:00
|
||||
custom:
|
||||
Author: haritamar
|
||||
Issue: "6246"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Features
|
||||
body: Exposures support metrics in lineage
|
||||
time: 2022-11-30T11:29:13.256034-05:00
|
||||
custom:
|
||||
Author: michelleark
|
||||
Issue: "6057"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Add support for Python 3.11
|
||||
time: 2022-12-06T15:07:04.753127+01:00
|
||||
custom:
|
||||
Author: joshuataylor MichelleArk jtcohen6
|
||||
Issue: "6147"
|
||||
PR: "6326"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Account for disabled flags on models in schema files more completely
|
||||
time: 2022-09-16T10:48:54.162273-05:00
|
||||
custom:
|
||||
Author: emmyoop
|
||||
Issue: "3992"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Add validation of enabled config for metrics, exposures and sources
|
||||
time: 2022-10-10T11:32:18.752322-05:00
|
||||
custom:
|
||||
Author: emmyoop
|
||||
Issue: "6030"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Fixes
|
||||
body: check length of args of python model function before accessing it
|
||||
time: 2022-10-11T16:07:15.464093-04:00
|
||||
custom:
|
||||
Author: chamini2
|
||||
Issue: "6041"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Add functors to ensure event types with str-type attributes are initialized
|
||||
to spec, even when provided non-str type params.
|
||||
time: 2022-10-16T17:37:42.846683-07:00
|
||||
custom:
|
||||
Author: versusfacit
|
||||
Issue: "5436"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Allow hooks to fail without halting execution flow
|
||||
time: 2022-11-07T09:53:14.340257-06:00
|
||||
custom:
|
||||
Author: ChenyuLInx
|
||||
Issue: "5625"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Clarify Error Message for how many models are allowed in a Python file
|
||||
time: 2022-11-15T08:10:21.527884-05:00
|
||||
custom:
|
||||
Author: justbldwn
|
||||
Issue: "6245"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: After this, will be possible to use default values for dbt.config.get
|
||||
time: 2022-11-24T16:34:19.039512764-03:00
|
||||
custom:
|
||||
Author: devmessias
|
||||
Issue: "6309"
|
||||
PR: "6317"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Use full path for writing manifest
|
||||
time: 2022-12-02T16:48:59.029519-05:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "6055"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Fixes
|
||||
body: '[CT-1284] Change Python model default materialization to table'
|
||||
time: 2022-12-13T11:26:20.550017-08:00
|
||||
custom:
|
||||
Author: aranke
|
||||
Issue: "5989"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Repair a regression which prevented basic logging before the logging subsystem
|
||||
is completely configured.
|
||||
time: 2022-12-14T15:53:07.396512-05:00
|
||||
custom:
|
||||
Author: peterallenwebb
|
||||
Issue: "6434"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Put black config in explicit config
|
||||
time: 2022-09-27T19:42:59.241433-07:00
|
||||
custom:
|
||||
Author: max-sixty
|
||||
Issue: "5946"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Added flat_graph attribute the Manifest class's deepcopy() coverage
|
||||
time: 2022-09-29T13:44:06.275941-04:00
|
||||
custom:
|
||||
Author: peterallenwebb
|
||||
Issue: "5809"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Add mypy configs so `mypy` passes from CLI
|
||||
time: 2022-10-05T12:03:10.061263-07:00
|
||||
custom:
|
||||
Author: max-sixty
|
||||
Issue: "5983"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Exception message cleanup.
|
||||
time: 2022-10-07T09:46:27.682872-05:00
|
||||
custom:
|
||||
Author: emmyoop
|
||||
Issue: "6023"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Add dmypy cache to gitignore
|
||||
time: 2022-10-07T14:00:44.227644-07:00
|
||||
custom:
|
||||
Author: max-sixty
|
||||
Issue: "6028"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Provide useful errors when the value of 'materialized' is invalid
|
||||
time: 2022-10-13T18:19:12.167548-04:00
|
||||
custom:
|
||||
Author: peterallenwebb
|
||||
Issue: "5229"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Fixed extra whitespace in strings introduced by black.
|
||||
time: 2022-10-17T15:15:11.499246-05:00
|
||||
custom:
|
||||
Author: luke-bassett
|
||||
Issue: "1350"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Clean up string formatting
|
||||
time: 2022-10-17T15:58:44.676549-04:00
|
||||
custom:
|
||||
Author: eve-johns
|
||||
Issue: "6068"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Remove the 'root_path' field from most nodes
|
||||
time: 2022-10-28T10:48:37.687886-04:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "6171"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Combine certain logging events with different levels
|
||||
time: 2022-10-28T11:03:44.887836-04:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "6173"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Convert threading tests to pytest
|
||||
time: 2022-11-08T07:45:50.589147-06:00
|
||||
custom:
|
||||
Author: stu-k
|
||||
Issue: "5942"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Convert postgres index tests to pytest
|
||||
time: 2022-11-08T11:56:33.743042-06:00
|
||||
custom:
|
||||
Author: stu-k
|
||||
Issue: "5770"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Convert use color tests to pytest
|
||||
time: 2022-11-08T13:31:04.788547-06:00
|
||||
custom:
|
||||
Author: stu-k
|
||||
Issue: "5771"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Add github actions workflow to generate high level CLI API docs
|
||||
time: 2022-11-16T13:00:37.916202-06:00
|
||||
custom:
|
||||
Author: stu-k
|
||||
Issue: "5942"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Functionality-neutral refactor of event logging system to improve encapsulation
|
||||
and modularity.
|
||||
time: 2022-11-18T14:57:17.792622-05:00
|
||||
custom:
|
||||
Author: peterallenwebb
|
||||
Issue: "6139"
|
||||
PR: "6291"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Consolidate ParsedNode and CompiledNode classes
|
||||
time: 2022-12-05T16:49:48.563583-05:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "6383"
|
||||
PR: "6384"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Prevent doc gen workflow from running on forks
|
||||
time: 2022-12-06T09:40:15.301984-06:00
|
||||
custom:
|
||||
Author: stu-k
|
||||
Issue: "6386"
|
||||
PR: "6390"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Fix intermittent database connection failure in Windows CI test
|
||||
time: 2022-12-06T11:30:53.166009-07:00
|
||||
custom:
|
||||
Author: MichelleArk dbeatty10
|
||||
Issue: "6394"
|
||||
PR: "6395"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Refactor and clean up manifest nodes
|
||||
time: 2022-12-11T21:42:40.560074-05:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "6426"
|
||||
PR: "6427"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Restore important legacy logging behaviors, following refactor which removed
|
||||
them
|
||||
time: 2022-12-13T21:41:06.815133-05:00
|
||||
custom:
|
||||
Author: peterallenwebb
|
||||
Issue: "6437"
|
||||
8
.changes/1.4.1.md
Normal file
8
.changes/1.4.1.md
Normal file
@@ -0,0 +1,8 @@
|
||||
## dbt-core 1.4.1 - January 26, 2023
|
||||
|
||||
### Fixes
|
||||
|
||||
- [Regression] exposure_content referenced incorrectly ([#6738](https://github.com/dbt-labs/dbt-core/issues/6738))
|
||||
|
||||
### Contributors
|
||||
- [@Mathyoub](https://github.com/Mathyoub) ([#6738](https://github.com/dbt-labs/dbt-core/issues/6738))
|
||||
20
.changes/1.4.2.md
Normal file
20
.changes/1.4.2.md
Normal file
@@ -0,0 +1,20 @@
|
||||
## dbt-core 1.4.2 - February 23, 2023
|
||||
|
||||
### Fixes
|
||||
|
||||
- Sort cli vars before hashing for partial parsing ([#6710](https://github.com/dbt-labs/dbt-core/issues/6710))
|
||||
- Remove pin on packaging and stop using it for prerelease comparisons ([#6834](https://github.com/dbt-labs/dbt-core/issues/6834))
|
||||
- Readd depends_on.macros to SeedNode, to support seeds with hooks calling macros ([#6806](https://github.com/dbt-labs/dbt-core/issues/6806))
|
||||
- Fix regression of --quiet cli parameter behavior ([#6749](https://github.com/dbt-labs/dbt-core/issues/6749))
|
||||
- Ensure results from hooks contain nodes when processing them ([#6796](https://github.com/dbt-labs/dbt-core/issues/6796))
|
||||
- Always flush stdout after logging ([#6901](https://github.com/dbt-labs/dbt-core/issues/6901))
|
||||
- Set relation_name in test nodes at compile time ([#6930](https://github.com/dbt-labs/dbt-core/issues/6930))
|
||||
- Fix disabled definition in WritableManifest ([#6752](https://github.com/dbt-labs/dbt-core/issues/6752))
|
||||
- Fix regression in logbook log output ([#7028](https://github.com/dbt-labs/dbt-core/issues/7028))
|
||||
|
||||
### Docs
|
||||
|
||||
- Fix JSON path to overview docs ([dbt-docs/#366](https://github.com/dbt-labs/dbt-docs/issues/366))
|
||||
|
||||
### Contributors
|
||||
- [@halvorlu](https://github.com/halvorlu) ([#366](https://github.com/dbt-labs/dbt-core/issues/366))
|
||||
5
.changes/1.4.3.md
Normal file
5
.changes/1.4.3.md
Normal file
@@ -0,0 +1,5 @@
|
||||
## dbt-core 1.4.3 - February 24, 2023
|
||||
|
||||
### Fixes
|
||||
|
||||
- Fix semver comparison logic by ensuring numeric values ([#7039](https://github.com/dbt-labs/dbt-core/issues/7039))
|
||||
8
.changes/1.4.4.md
Normal file
8
.changes/1.4.4.md
Normal file
@@ -0,0 +1,8 @@
|
||||
## dbt-core 1.4.4 - February 28, 2023
|
||||
|
||||
### Fixes
|
||||
|
||||
- add pytz dependency ([#7077](https://github.com/dbt-labs/dbt-core/issues/7077))
|
||||
|
||||
### Contributors
|
||||
- [@sdebruyn](https://github.com/sdebruyn) ([#7077](https://github.com/dbt-labs/dbt-core/issues/7077))
|
||||
19
.changes/1.4.5.md
Normal file
19
.changes/1.4.5.md
Normal file
@@ -0,0 +1,19 @@
|
||||
## dbt-core 1.4.5 - March 10, 2023
|
||||
|
||||
### Fixes
|
||||
|
||||
- Fix compilation logic for ephemeral nodes ([#6885](https://github.com/dbt-labs/dbt-core/issues/6885))
|
||||
- allow adapters to change model name resolution in py models ([#7114](https://github.com/dbt-labs/dbt-core/issues/7114))
|
||||
|
||||
### Docs
|
||||
|
||||
- Fix JSON path to package overview docs ([dbt-docs/#390](https://github.com/dbt-labs/dbt-docs/issues/390))
|
||||
|
||||
### Under the Hood
|
||||
|
||||
- Moving simple_seed to adapter zone to help adapter test conversions ([#CT-1959](https://github.com/dbt-labs/dbt-core/issues/CT-1959))
|
||||
|
||||
### Contributors
|
||||
- [@dbeatty10](https://github.com/dbeatty10) ([#390](https://github.com/dbt-labs/dbt-core/issues/390))
|
||||
- [@nssalian](https://github.com/nssalian) ([#CT-1959](https://github.com/dbt-labs/dbt-core/issues/CT-1959))
|
||||
- [@rlh1994](https://github.com/rlh1994) ([#390](https://github.com/dbt-labs/dbt-core/issues/390))
|
||||
@@ -1,9 +0,0 @@
|
||||
kind: Breaking Changes
|
||||
body: Cleaned up exceptions to directly raise in code. Also updated the existing
|
||||
exception to meet PEP guidelines.Removed use of all exception
|
||||
functions in the code base and marked them all as deprecated to be removed next
|
||||
minor release.
|
||||
time: 2022-12-05T14:19:37.863032-06:00
|
||||
custom:
|
||||
Author: emmyoop
|
||||
Issue: 6339 6393 6460
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: "Dependencies"
|
||||
body: "Update agate requirement from <1.6.4,>=1.6 to >=1.6,<1.7.1 in /core"
|
||||
time: 2023-01-04T00:03:06.00000Z
|
||||
custom:
|
||||
Author: dependabot[bot]
|
||||
PR: 6506
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Docs
|
||||
body: Updated minor typos encountered when skipping profile setup
|
||||
time: 2023-01-02T17:02:16.66596191-05:00
|
||||
custom:
|
||||
Author: nshuman1
|
||||
Issue: 6529
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Making timestamp optional for metrics
|
||||
time: 2022-12-07T09:17:22.571877-06:00
|
||||
custom:
|
||||
Author: callum-mcdata
|
||||
Issue: "6398"
|
||||
PR: "9400"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: The meta configuration field is now included in the node_info property of structured
|
||||
logs.
|
||||
time: 2022-12-30T10:48:20.486416-06:00
|
||||
custom:
|
||||
Author: tmastny
|
||||
Issue: "6216"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Features
|
||||
body: Adds buildable selection mode
|
||||
time: 2023-01-02T09:13:35.663627-05:00
|
||||
custom:
|
||||
Author: agpapa
|
||||
Issue: "6365"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: '--warn-error-options: Treat warnings as errors for specific events, based on
|
||||
user configuration'
|
||||
time: 2023-01-04T18:10:03.203142-05:00
|
||||
custom:
|
||||
Author: MichelleArk
|
||||
Issue: "6165"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: fix missing f-strings, convert old .format() messages to f-strings for consistency
|
||||
time: 2022-11-13T10:41:50.009727-05:00
|
||||
custom:
|
||||
Author: justbldwn
|
||||
Issue: "6241"
|
||||
PR: "6243"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Fix typo in util.py
|
||||
time: 2022-11-17T22:03:20.4836855+09:00
|
||||
custom:
|
||||
Author: eltociear
|
||||
Issue: "4904"
|
||||
PR: "6037"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: add pre-commit install to make dev script in Makefile
|
||||
time: 2022-12-12T11:59:12.175136-05:00
|
||||
custom:
|
||||
Author: justbldwn
|
||||
Issue: "6269"
|
||||
PR: "6417"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Late-rendering for `pre_` and `post_hook`s in `dbt_project.yml`
|
||||
time: 2022-12-13T09:26:55.11397-07:00
|
||||
custom:
|
||||
Author: dbeatty10
|
||||
Issue: "6411"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Fixes
|
||||
body: '[CT-1591] Don''t parse empty Python files'
|
||||
time: 2022-12-13T11:39:15.818464-08:00
|
||||
custom:
|
||||
Author: aranke
|
||||
Issue: "6345"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Fixes
|
||||
body: fix docs generate --defer by adding defer_to_manifest to before_run
|
||||
time: 2022-12-26T01:02:11.630614+01:00
|
||||
custom:
|
||||
Author: mivanicova
|
||||
Issue: "6488"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Bug when partial parsing with an empty schema file
|
||||
time: 2023-01-01T22:34:05.97322-05:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "4850"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Fix DBT_FAVOR_STATE env var
|
||||
time: 2023-01-04T14:10:47.637495-08:00
|
||||
custom:
|
||||
Author: NiallRees
|
||||
Issue: "5859"
|
||||
PR: "6392"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Restore historical behavior of certain disabled test messages, so that they
|
||||
are at the less obtrusive debug level, rather than the warning level.
|
||||
time: 2023-01-09T16:12:54.064875-05:00
|
||||
custom:
|
||||
Author: peterallenwebb
|
||||
Issue: "6501"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Bump mashumuro version to get regression fix and add unit test to verify that
|
||||
fix.
|
||||
time: 2023-01-10T12:41:32.339631-05:00
|
||||
custom:
|
||||
Author: peterallenwebb
|
||||
Issue: "6428"
|
||||
6
.changes/unreleased/Fixes-20230215-104536.yaml
Normal file
6
.changes/unreleased/Fixes-20230215-104536.yaml
Normal file
@@ -0,0 +1,6 @@
|
||||
kind: Fixes
|
||||
body: Make use of hashlib.md5() FIPS compliant
|
||||
time: 2023-02-15T10:45:36.755797+01:00
|
||||
custom:
|
||||
Author: nielspardon
|
||||
Issue: "6900"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Treat dense text blobs as binary for `git grep`
|
||||
time: 2022-12-19T19:34:35.890275-07:00
|
||||
custom:
|
||||
Author: dbeatty10
|
||||
Issue: "6294"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Prune partial parsing logging events
|
||||
time: 2022-12-21T12:19:04.7402-05:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "6313"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Updating the deprecation warning in the metric attributes renamed event
|
||||
time: 2023-01-04T15:52:57.916398-06:00
|
||||
custom:
|
||||
Author: callum-mcdata
|
||||
Issue: "6507"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: '[CT-1693] Port severity test to Pytest'
|
||||
time: 2023-01-06T11:28:55.800547-08:00
|
||||
custom:
|
||||
Author: aranke
|
||||
Issue: "6466"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: '[CT-1694] Deprecate event tracking tests'
|
||||
time: 2023-01-06T11:44:12.210821-08:00
|
||||
custom:
|
||||
Author: aranke
|
||||
Issue: "6467"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Reorganize structured logging events to have two top keys
|
||||
time: 2023-01-09T09:59:07.842187-05:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "6311"
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Combine some logging events
|
||||
time: 2023-01-10T11:42:33.580756-05:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: 1716 1717 1719
|
||||
@@ -1,6 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Check length of escaped strings in the adapter test
|
||||
time: 2023-01-10T14:56:48.044198-07:00
|
||||
custom:
|
||||
Author: dbeatty10
|
||||
Issue: "6566"
|
||||
6
.changes/unreleased/Under the Hood-20230324-144050.yaml
Normal file
6
.changes/unreleased/Under the Hood-20230324-144050.yaml
Normal file
@@ -0,0 +1,6 @@
|
||||
kind: Under the Hood
|
||||
body: Remove upper pin for hologram/jsonschema
|
||||
time: 2023-03-24T14:40:50.574108-04:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "6775"
|
||||
@@ -97,22 +97,28 @@ footerFormat: |
|
||||
{{- /* we only want to include non-core team contributors */}}
|
||||
{{- if not (has $authorLower $core_team)}}
|
||||
{{- $changeList := splitList " " $change.Custom.Author }}
|
||||
{{- /* Docs kind link back to dbt-docs instead of dbt-core issues */}}
|
||||
{{- $IssueList := list }}
|
||||
{{- $changeLink := $change.Kind }}
|
||||
{{- if or (eq $change.Kind "Dependencies") (eq $change.Kind "Security") }}
|
||||
{{- $changeLink = "[#nbr](https://github.com/dbt-labs/dbt-core/pull/nbr)" | replace "nbr" $change.Custom.PR }}
|
||||
{{- else if eq $change.Kind "Docs"}}
|
||||
{{- $changeLink = "[dbt-docs/#nbr](https://github.com/dbt-labs/dbt-docs/issues/nbr)" | replace "nbr" $change.Custom.Issue }}
|
||||
{{- $changes := splitList " " $change.Custom.PR }}
|
||||
{{- range $issueNbr := $changes }}
|
||||
{{- $changeLink := "[#nbr](https://github.com/dbt-labs/dbt-core/pull/nbr)" | replace "nbr" $issueNbr }}
|
||||
{{- $IssueList = append $IssueList $changeLink }}
|
||||
{{- end -}}
|
||||
{{- else }}
|
||||
{{- $changeLink = "[#nbr](https://github.com/dbt-labs/dbt-core/issues/nbr)" | replace "nbr" $change.Custom.Issue }}
|
||||
{{- $changes := splitList " " $change.Custom.Issue }}
|
||||
{{- range $issueNbr := $changes }}
|
||||
{{- $changeLink := "[#nbr](https://github.com/dbt-labs/dbt-core/issues/nbr)" | replace "nbr" $issueNbr }}
|
||||
{{- $IssueList = append $IssueList $changeLink }}
|
||||
{{- end -}}
|
||||
{{- end }}
|
||||
{{- /* check if this contributor has other changes associated with them already */}}
|
||||
{{- if hasKey $contributorDict $author }}
|
||||
{{- $contributionList := get $contributorDict $author }}
|
||||
{{- $contributionList = append $contributionList $changeLink }}
|
||||
{{- $contributionList = concat $contributionList $IssueList }}
|
||||
{{- $contributorDict := set $contributorDict $author $contributionList }}
|
||||
{{- else }}
|
||||
{{- $contributionList := list $changeLink }}
|
||||
{{- $contributionList := $IssueList }}
|
||||
{{- $contributorDict := set $contributorDict $author $contributionList }}
|
||||
{{- end }}
|
||||
{{- end}}
|
||||
|
||||
109
.github/workflows/nightly-release.yml
vendored
Normal file
109
.github/workflows/nightly-release.yml
vendored
Normal file
@@ -0,0 +1,109 @@
|
||||
# **what?**
|
||||
# Nightly releases to GitHub and PyPI. This workflow produces the following outcome:
|
||||
# - generate and validate data for night release (commit SHA, version number, release branch);
|
||||
# - pass data to release workflow;
|
||||
# - night release will be pushed to GitHub as a draft release;
|
||||
# - night build will be pushed to test PyPI;
|
||||
#
|
||||
# **why?**
|
||||
# Ensure an automated and tested release process for nightly builds
|
||||
#
|
||||
# **when?**
|
||||
# This workflow runs on schedule or can be run manually on demand.
|
||||
|
||||
name: Nightly Test Release to GitHub and PyPI
|
||||
|
||||
on:
|
||||
workflow_dispatch: # for manual triggering
|
||||
schedule:
|
||||
- cron: 0 9 * * *
|
||||
|
||||
permissions:
|
||||
contents: write # this is the permission that allows creating a new release
|
||||
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
|
||||
env:
|
||||
RELEASE_BRANCH: "main"
|
||||
|
||||
jobs:
|
||||
aggregate-release-data:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
outputs:
|
||||
commit_sha: ${{ steps.resolve-commit-sha.outputs.release_commit }}
|
||||
version_number: ${{ steps.nightly-release-version.outputs.number }}
|
||||
release_branch: ${{ steps.release-branch.outputs.name }}
|
||||
|
||||
steps:
|
||||
- name: "Checkout ${{ github.repository }} Branch ${{ env.RELEASE_BRANCH }}"
|
||||
uses: actions/checkout@v3
|
||||
with:
|
||||
ref: ${{ env.RELEASE_BRANCH }}
|
||||
|
||||
- name: "Resolve Commit To Release"
|
||||
id: resolve-commit-sha
|
||||
run: |
|
||||
commit_sha=$(git rev-parse HEAD)
|
||||
echo "release_commit=$commit_sha" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: "Get Current Version Number"
|
||||
id: version-number-sources
|
||||
run: |
|
||||
current_version=`awk -F"current_version = " '{print $2}' .bumpversion.cfg | tr '\n' ' '`
|
||||
echo "current_version=$current_version" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: "Audit Version And Parse Into Parts"
|
||||
id: semver
|
||||
uses: dbt-labs/actions/parse-semver@v1.1.0
|
||||
with:
|
||||
version: ${{ steps.version-number-sources.outputs.current_version }}
|
||||
|
||||
- name: "Get Current Date"
|
||||
id: current-date
|
||||
run: echo "date=$(date +'%m%d%Y')" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: "Generate Nightly Release Version Number"
|
||||
id: nightly-release-version
|
||||
run: |
|
||||
number="${{ steps.semver.outputs.version }}.dev${{ steps.current-date.outputs.date }}+nightly"
|
||||
echo "number=$number" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: "Audit Nightly Release Version And Parse Into Parts"
|
||||
uses: dbt-labs/actions/parse-semver@v1.1.0
|
||||
with:
|
||||
version: ${{ steps.nightly-release-version.outputs.number }}
|
||||
|
||||
- name: "Set Release Branch"
|
||||
id: release-branch
|
||||
run: |
|
||||
echo "name=${{ env.RELEASE_BRANCH }}" >> $GITHUB_OUTPUT
|
||||
|
||||
log-outputs-aggregate-release-data:
|
||||
runs-on: ubuntu-latest
|
||||
needs: [aggregate-release-data]
|
||||
|
||||
steps:
|
||||
- name: "[DEBUG] Log Outputs"
|
||||
run: |
|
||||
echo commit_sha : ${{ needs.aggregate-release-data.outputs.commit_sha }}
|
||||
echo version_number: ${{ needs.aggregate-release-data.outputs.version_number }}
|
||||
echo release_branch: ${{ needs.aggregate-release-data.outputs.release_branch }}
|
||||
|
||||
release-github-pypi:
|
||||
needs: [aggregate-release-data]
|
||||
|
||||
uses: ./.github/workflows/release.yml
|
||||
with:
|
||||
sha: ${{ needs.aggregate-release-data.outputs.commit_sha }}
|
||||
target_branch: ${{ needs.aggregate-release-data.outputs.release-branch }}
|
||||
version_number: ${{ needs.aggregate-release-data.outputs.version_number }}
|
||||
build_script_path: "scripts/build-dist.sh"
|
||||
env_setup_script_path: "scripts/env-setup.sh"
|
||||
s3_bucket_name: "core-team-artifacts"
|
||||
package_test_command: "dbt --version"
|
||||
test_run: true
|
||||
nightly_release: true
|
||||
secrets: inherit
|
||||
339
.github/workflows/release.yml
vendored
339
.github/workflows/release.yml
vendored
@@ -1,24 +1,110 @@
|
||||
# **what?**
|
||||
# Take the given commit, run unit tests specifically on that sha, build and
|
||||
# package it, and then release to GitHub and PyPi with that specific build
|
||||
|
||||
# Release workflow provides the following steps:
|
||||
# - checkout the given commit;
|
||||
# - validate version in sources and changelog file for given version;
|
||||
# - bump the version and generate a changelog if needed;
|
||||
# - merge all changes to the target branch if needed;
|
||||
# - run unit and integration tests against given commit;
|
||||
# - build and package that SHA;
|
||||
# - release it to GitHub and PyPI with that specific build;
|
||||
#
|
||||
# **why?**
|
||||
# Ensure an automated and tested release process
|
||||
|
||||
#
|
||||
# **when?**
|
||||
# This will only run manually with a given sha and version
|
||||
# This workflow can be run manually on demand or can be called by other workflows
|
||||
|
||||
name: Release to GitHub and PyPi
|
||||
name: Release to GitHub and PyPI
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
sha:
|
||||
description: 'The last commit sha in the release'
|
||||
required: true
|
||||
description: "The last commit sha in the release"
|
||||
type: string
|
||||
required: true
|
||||
target_branch:
|
||||
description: "The branch to release from"
|
||||
type: string
|
||||
required: true
|
||||
version_number:
|
||||
description: 'The release version number (i.e. 1.0.0b1)'
|
||||
required: true
|
||||
description: "The release version number (i.e. 1.0.0b1)"
|
||||
type: string
|
||||
required: true
|
||||
build_script_path:
|
||||
description: "Build script path"
|
||||
type: string
|
||||
default: "scripts/build-dist.sh"
|
||||
required: true
|
||||
env_setup_script_path:
|
||||
description: "Environment setup script path"
|
||||
type: string
|
||||
default: "scripts/env-setup.sh"
|
||||
required: false
|
||||
s3_bucket_name:
|
||||
description: "AWS S3 bucket name"
|
||||
type: string
|
||||
default: "core-team-artifacts"
|
||||
required: true
|
||||
package_test_command:
|
||||
description: "Package test command"
|
||||
type: string
|
||||
default: "dbt --version"
|
||||
required: true
|
||||
test_run:
|
||||
description: "Test run (Publish release as draft)"
|
||||
type: boolean
|
||||
default: true
|
||||
required: false
|
||||
nightly_release:
|
||||
description: "Nightly release to dev environment"
|
||||
type: boolean
|
||||
default: false
|
||||
required: false
|
||||
workflow_call:
|
||||
inputs:
|
||||
sha:
|
||||
description: "The last commit sha in the release"
|
||||
type: string
|
||||
required: true
|
||||
target_branch:
|
||||
description: "The branch to release from"
|
||||
type: string
|
||||
required: true
|
||||
version_number:
|
||||
description: "The release version number (i.e. 1.0.0b1)"
|
||||
type: string
|
||||
required: true
|
||||
build_script_path:
|
||||
description: "Build script path"
|
||||
type: string
|
||||
default: "scripts/build-dist.sh"
|
||||
required: true
|
||||
env_setup_script_path:
|
||||
description: "Environment setup script path"
|
||||
type: string
|
||||
default: "scripts/env-setup.sh"
|
||||
required: false
|
||||
s3_bucket_name:
|
||||
description: "AWS S3 bucket name"
|
||||
type: string
|
||||
default: "core-team-artifacts"
|
||||
required: true
|
||||
package_test_command:
|
||||
description: "Package test command"
|
||||
type: string
|
||||
default: "dbt --version"
|
||||
required: true
|
||||
test_run:
|
||||
description: "Test run (Publish release as draft)"
|
||||
type: boolean
|
||||
default: true
|
||||
required: false
|
||||
nightly_release:
|
||||
description: "Nightly release to dev environment"
|
||||
type: boolean
|
||||
default: false
|
||||
required: false
|
||||
|
||||
permissions:
|
||||
contents: write # this is the permission that allows creating a new release
|
||||
@@ -28,175 +114,116 @@ defaults:
|
||||
shell: bash
|
||||
|
||||
jobs:
|
||||
unit:
|
||||
name: Unit test
|
||||
|
||||
log-inputs:
|
||||
name: Log Inputs
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
env:
|
||||
TOXENV: "unit"
|
||||
|
||||
steps:
|
||||
- name: Check out the repository
|
||||
uses: actions/checkout@v2
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: ${{ github.event.inputs.sha }}
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v2
|
||||
with:
|
||||
python-version: 3.8
|
||||
|
||||
- name: Install python dependencies
|
||||
- name: "[DEBUG] Print Variables"
|
||||
run: |
|
||||
pip install --user --upgrade pip
|
||||
pip install tox
|
||||
pip --version
|
||||
tox --version
|
||||
echo The last commit sha in the release: ${{ inputs.sha }}
|
||||
echo The branch to release from: ${{ inputs.target_branch }}
|
||||
echo The release version number: ${{ inputs.version_number }}
|
||||
echo Build script path: ${{ inputs.build_script_path }}
|
||||
echo Environment setup script path: ${{ inputs.env_setup_script_path }}
|
||||
echo AWS S3 bucket name: ${{ inputs.s3_bucket_name }}
|
||||
echo Package test command: ${{ inputs.package_test_command }}
|
||||
echo Test run: ${{ inputs.test_run }}
|
||||
echo Nightly release: ${{ inputs.nightly_release }}
|
||||
|
||||
- name: Run tox
|
||||
run: tox
|
||||
bump-version-generate-changelog:
|
||||
name: Bump package version, Generate changelog
|
||||
|
||||
build:
|
||||
name: build packages
|
||||
uses: dbt-labs/dbt-release/.github/workflows/release-prep.yml@main
|
||||
|
||||
with:
|
||||
sha: ${{ inputs.sha }}
|
||||
version_number: ${{ inputs.version_number }}
|
||||
target_branch: ${{ inputs.target_branch }}
|
||||
env_setup_script_path: ${{ inputs.env_setup_script_path }}
|
||||
test_run: ${{ inputs.test_run }}
|
||||
nightly_release: ${{ inputs.nightly_release }}
|
||||
|
||||
secrets: inherit
|
||||
|
||||
log-outputs-bump-version-generate-changelog:
|
||||
name: "[Log output] Bump package version, Generate changelog"
|
||||
if: ${{ !failure() && !cancelled() }}
|
||||
|
||||
needs: [bump-version-generate-changelog]
|
||||
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Check out the repository
|
||||
uses: actions/checkout@v2
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: ${{ github.event.inputs.sha }}
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v2
|
||||
with:
|
||||
python-version: 3.8
|
||||
|
||||
- name: Install python dependencies
|
||||
- name: Print variables
|
||||
run: |
|
||||
pip install --user --upgrade pip
|
||||
pip install --upgrade setuptools wheel twine check-wheel-contents
|
||||
pip --version
|
||||
echo Final SHA : ${{ needs.bump-version-generate-changelog.outputs.final_sha }}
|
||||
echo Changelog path: ${{ needs.bump-version-generate-changelog.outputs.changelog_path }}
|
||||
|
||||
- name: Build distributions
|
||||
run: ./scripts/build-dist.sh
|
||||
build-test-package:
|
||||
name: Build, Test, Package
|
||||
if: ${{ !failure() && !cancelled() }}
|
||||
needs: [bump-version-generate-changelog]
|
||||
|
||||
- name: Show distributions
|
||||
run: ls -lh dist/
|
||||
uses: dbt-labs/dbt-release/.github/workflows/build.yml@main
|
||||
|
||||
- name: Check distribution descriptions
|
||||
run: |
|
||||
twine check dist/*
|
||||
with:
|
||||
sha: ${{ needs.bump-version-generate-changelog.outputs.final_sha }}
|
||||
version_number: ${{ inputs.version_number }}
|
||||
changelog_path: ${{ needs.bump-version-generate-changelog.outputs.changelog_path }}
|
||||
build_script_path: ${{ inputs.build_script_path }}
|
||||
s3_bucket_name: ${{ inputs.s3_bucket_name }}
|
||||
package_test_command: ${{ inputs.package_test_command }}
|
||||
test_run: ${{ inputs.test_run }}
|
||||
nightly_release: ${{ inputs.nightly_release }}
|
||||
|
||||
- name: Check wheel contents
|
||||
run: |
|
||||
check-wheel-contents dist/*.whl --ignore W007,W008
|
||||
|
||||
- uses: actions/upload-artifact@v2
|
||||
with:
|
||||
name: dist
|
||||
path: |
|
||||
dist/
|
||||
!dist/dbt-${{github.event.inputs.version_number}}.tar.gz
|
||||
|
||||
test-build:
|
||||
name: verify packages
|
||||
|
||||
needs: [build, unit]
|
||||
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v2
|
||||
with:
|
||||
python-version: 3.8
|
||||
|
||||
- name: Install python dependencies
|
||||
run: |
|
||||
pip install --user --upgrade pip
|
||||
pip install --upgrade wheel
|
||||
pip --version
|
||||
|
||||
- uses: actions/download-artifact@v2
|
||||
with:
|
||||
name: dist
|
||||
path: dist/
|
||||
|
||||
- name: Show distributions
|
||||
run: ls -lh dist/
|
||||
|
||||
- name: Install wheel distributions
|
||||
run: |
|
||||
find ./dist/*.whl -maxdepth 1 -type f | xargs pip install --force-reinstall --find-links=dist/
|
||||
|
||||
- name: Check wheel distributions
|
||||
run: |
|
||||
dbt --version
|
||||
|
||||
- name: Install source distributions
|
||||
run: |
|
||||
find ./dist/*.gz -maxdepth 1 -type f | xargs pip install --force-reinstall --find-links=dist/
|
||||
|
||||
- name: Check source distributions
|
||||
run: |
|
||||
dbt --version
|
||||
secrets:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
|
||||
github-release:
|
||||
name: GitHub Release
|
||||
if: ${{ !failure() && !cancelled() }}
|
||||
|
||||
needs: test-build
|
||||
needs: [bump-version-generate-changelog, build-test-package]
|
||||
|
||||
runs-on: ubuntu-latest
|
||||
uses: dbt-labs/dbt-release/.github/workflows/github-release.yml@main
|
||||
|
||||
steps:
|
||||
- uses: actions/download-artifact@v2
|
||||
with:
|
||||
name: dist
|
||||
path: '.'
|
||||
|
||||
# Need to set an output variable because env variables can't be taken as input
|
||||
# This is needed for the next step with releasing to GitHub
|
||||
- name: Find release type
|
||||
id: release_type
|
||||
env:
|
||||
IS_PRERELEASE: ${{ contains(github.event.inputs.version_number, 'rc') || contains(github.event.inputs.version_number, 'b') }}
|
||||
run: |
|
||||
echo ::set-output name=isPrerelease::$IS_PRERELEASE
|
||||
|
||||
- name: Creating GitHub Release
|
||||
uses: softprops/action-gh-release@v1
|
||||
with:
|
||||
name: dbt-core v${{github.event.inputs.version_number}}
|
||||
tag_name: v${{github.event.inputs.version_number}}
|
||||
prerelease: ${{ steps.release_type.outputs.isPrerelease }}
|
||||
target_commitish: ${{github.event.inputs.sha}}
|
||||
body: |
|
||||
[Release notes](https://github.com/dbt-labs/dbt-core/blob/main/CHANGELOG.md)
|
||||
files: |
|
||||
dbt_postgres-${{github.event.inputs.version_number}}-py3-none-any.whl
|
||||
dbt_core-${{github.event.inputs.version_number}}-py3-none-any.whl
|
||||
dbt-postgres-${{github.event.inputs.version_number}}.tar.gz
|
||||
dbt-core-${{github.event.inputs.version_number}}.tar.gz
|
||||
with:
|
||||
sha: ${{ needs.bump-version-generate-changelog.outputs.final_sha }}
|
||||
version_number: ${{ inputs.version_number }}
|
||||
changelog_path: ${{ needs.bump-version-generate-changelog.outputs.changelog_path }}
|
||||
test_run: ${{ inputs.test_run }}
|
||||
|
||||
pypi-release:
|
||||
name: Pypi release
|
||||
name: PyPI Release
|
||||
|
||||
runs-on: ubuntu-latest
|
||||
needs: [github-release]
|
||||
|
||||
needs: github-release
|
||||
uses: dbt-labs/dbt-release/.github/workflows/pypi-release.yml@main
|
||||
|
||||
environment: PypiProd
|
||||
steps:
|
||||
- uses: actions/download-artifact@v2
|
||||
with:
|
||||
name: dist
|
||||
path: 'dist'
|
||||
with:
|
||||
version_number: ${{ inputs.version_number }}
|
||||
test_run: ${{ inputs.test_run }}
|
||||
|
||||
- name: Publish distribution to PyPI
|
||||
uses: pypa/gh-action-pypi-publish@v1.4.2
|
||||
with:
|
||||
password: ${{ secrets.PYPI_API_TOKEN }}
|
||||
secrets:
|
||||
PYPI_API_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
|
||||
TEST_PYPI_API_TOKEN: ${{ secrets.TEST_PYPI_API_TOKEN }}
|
||||
|
||||
slack-notification:
|
||||
name: Slack Notification
|
||||
if: ${{ failure() && (!inputs.test_run || inputs.nightly_release) }}
|
||||
|
||||
needs:
|
||||
[
|
||||
bump-version-generate-changelog,
|
||||
build-test-package,
|
||||
github-release,
|
||||
pypi-release,
|
||||
]
|
||||
|
||||
uses: dbt-labs/dbt-release/.github/workflows/slack-post-notification.yml@main
|
||||
with:
|
||||
status: "failure"
|
||||
|
||||
secrets:
|
||||
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_DEV_CORE_ALERTS }}
|
||||
|
||||
127
CHANGELOG.md
127
CHANGELOG.md
@@ -5,12 +5,83 @@
|
||||
- "Breaking changes" listed under a version may require action from end users or external maintainers when upgrading to that version.
|
||||
- Do not edit this file directly. This file is auto-generated using [changie](https://github.com/miniscruff/changie). For details on how to document a change, see [the contributing guide](https://github.com/dbt-labs/dbt-core/blob/main/CONTRIBUTING.md#adding-changelog-entry)
|
||||
|
||||
## dbt-core 1.4.0-b1 - December 15, 2022
|
||||
## dbt-core 1.4.5 - March 10, 2023
|
||||
|
||||
### Fixes
|
||||
|
||||
- Fix compilation logic for ephemeral nodes ([#6885](https://github.com/dbt-labs/dbt-core/issues/6885))
|
||||
- allow adapters to change model name resolution in py models ([#7114](https://github.com/dbt-labs/dbt-core/issues/7114))
|
||||
|
||||
### Docs
|
||||
|
||||
- Fix JSON path to package overview docs ([dbt-docs/#390](https://github.com/dbt-labs/dbt-docs/issues/390))
|
||||
|
||||
### Under the Hood
|
||||
|
||||
- Moving simple_seed to adapter zone to help adapter test conversions ([#CT-1959](https://github.com/dbt-labs/dbt-core/issues/CT-1959))
|
||||
|
||||
### Contributors
|
||||
- [@dbeatty10](https://github.com/dbeatty10) ([#390](https://github.com/dbt-labs/dbt-core/issues/390))
|
||||
- [@nssalian](https://github.com/nssalian) ([#CT-1959](https://github.com/dbt-labs/dbt-core/issues/CT-1959))
|
||||
- [@rlh1994](https://github.com/rlh1994) ([#390](https://github.com/dbt-labs/dbt-core/issues/390))
|
||||
|
||||
|
||||
## dbt-core 1.4.4 - February 28, 2023
|
||||
|
||||
### Fixes
|
||||
|
||||
- add pytz dependency ([#7077](https://github.com/dbt-labs/dbt-core/issues/7077))
|
||||
|
||||
### Contributors
|
||||
- [@sdebruyn](https://github.com/sdebruyn) ([#7077](https://github.com/dbt-labs/dbt-core/issues/7077))
|
||||
|
||||
## dbt-core 1.4.3 - February 24, 2023
|
||||
|
||||
### Fixes
|
||||
|
||||
- Fix semver comparison logic by ensuring numeric values ([#7039](https://github.com/dbt-labs/dbt-core/issues/7039))
|
||||
|
||||
## dbt-core 1.4.2 - February 23, 2023
|
||||
|
||||
### Fixes
|
||||
|
||||
- Sort cli vars before hashing for partial parsing ([#6710](https://github.com/dbt-labs/dbt-core/issues/6710))
|
||||
- Remove pin on packaging and stop using it for prerelease comparisons ([#6834](https://github.com/dbt-labs/dbt-core/issues/6834))
|
||||
- Readd depends_on.macros to SeedNode, to support seeds with hooks calling macros ([#6806](https://github.com/dbt-labs/dbt-core/issues/6806))
|
||||
- Fix regression of --quiet cli parameter behavior ([#6749](https://github.com/dbt-labs/dbt-core/issues/6749))
|
||||
- Ensure results from hooks contain nodes when processing them ([#6796](https://github.com/dbt-labs/dbt-core/issues/6796))
|
||||
- Always flush stdout after logging ([#6901](https://github.com/dbt-labs/dbt-core/issues/6901))
|
||||
- Set relation_name in test nodes at compile time ([#6930](https://github.com/dbt-labs/dbt-core/issues/6930))
|
||||
- Fix disabled definition in WritableManifest ([#6752](https://github.com/dbt-labs/dbt-core/issues/6752))
|
||||
- Fix regression in logbook log output ([#7028](https://github.com/dbt-labs/dbt-core/issues/7028))
|
||||
|
||||
### Docs
|
||||
|
||||
- Fix JSON path to overview docs ([dbt-docs/#366](https://github.com/dbt-labs/dbt-docs/issues/366))
|
||||
|
||||
### Contributors
|
||||
- [@halvorlu](https://github.com/halvorlu) ([#366](https://github.com/dbt-labs/dbt-core/issues/366))
|
||||
|
||||
## dbt-core 1.4.1 - January 26, 2023
|
||||
|
||||
### Fixes
|
||||
|
||||
- [Regression] exposure_content referenced incorrectly ([#6738](https://github.com/dbt-labs/dbt-core/issues/6738))
|
||||
|
||||
### Contributors
|
||||
- [@Mathyoub](https://github.com/Mathyoub) ([#6738](https://github.com/dbt-labs/dbt-core/issues/6738))
|
||||
|
||||
## dbt-core 1.4.0 - January 25, 2023
|
||||
|
||||
### Breaking Changes
|
||||
|
||||
- Cleaned up exceptions to directly raise in code. Also updated the existing exception to meet PEP guidelines.Removed use of all exception functions in the code base and marked them all as deprecated to be removed next minor release. ([#6339](https://github.com/dbt-labs/dbt-core/issues/6339), [#6393](https://github.com/dbt-labs/dbt-core/issues/6393), [#6460](https://github.com/dbt-labs/dbt-core/issues/6460))
|
||||
|
||||
### Features
|
||||
|
||||
- Added favor-state flag to optionally favor state nodes even if unselected node exists ([#2968](https://github.com/dbt-labs/dbt-core/issues/2968))
|
||||
- Added favor-state flag to optionally favor state nodes even if unselected node exists ([#5016](https://github.com/dbt-labs/dbt-core/issues/5016))
|
||||
- Update structured logging. Convert to using protobuf messages. Ensure events are enriched with node_info. ([#5610](https://github.com/dbt-labs/dbt-core/issues/5610))
|
||||
- incremental predicates ([#5680](https://github.com/dbt-labs/dbt-core/issues/5680))
|
||||
- Friendlier error messages when packages.yml is malformed ([#5486](https://github.com/dbt-labs/dbt-core/issues/5486))
|
||||
- Allow partitions in external tables to be supplied as a list ([#5929](https://github.com/dbt-labs/dbt-core/issues/5929))
|
||||
- extend -f flag shorthand for seed command ([#5990](https://github.com/dbt-labs/dbt-core/issues/5990))
|
||||
@@ -19,7 +90,10 @@
|
||||
- Added an md5 function to the base context ([#6246](https://github.com/dbt-labs/dbt-core/issues/6246))
|
||||
- Exposures support metrics in lineage ([#6057](https://github.com/dbt-labs/dbt-core/issues/6057))
|
||||
- Add support for Python 3.11 ([#6147](https://github.com/dbt-labs/dbt-core/issues/6147))
|
||||
- incremental predicates ([#5680](https://github.com/dbt-labs/dbt-core/issues/5680))
|
||||
- Making timestamp optional for metrics ([#6398](https://github.com/dbt-labs/dbt-core/issues/6398))
|
||||
- The meta configuration field is now included in the node_info property of structured logs. ([#6216](https://github.com/dbt-labs/dbt-core/issues/6216))
|
||||
- Adds buildable selection mode ([#6365](https://github.com/dbt-labs/dbt-core/issues/6365))
|
||||
- --warn-error-options: Treat warnings as errors for specific events, based on user configuration ([#6165](https://github.com/dbt-labs/dbt-core/issues/6165))
|
||||
|
||||
### Fixes
|
||||
|
||||
@@ -28,11 +102,25 @@
|
||||
- check length of args of python model function before accessing it ([#6041](https://github.com/dbt-labs/dbt-core/issues/6041))
|
||||
- Add functors to ensure event types with str-type attributes are initialized to spec, even when provided non-str type params. ([#5436](https://github.com/dbt-labs/dbt-core/issues/5436))
|
||||
- Allow hooks to fail without halting execution flow ([#5625](https://github.com/dbt-labs/dbt-core/issues/5625))
|
||||
- fix missing f-strings, convert old .format() messages to f-strings for consistency ([#6241](https://github.com/dbt-labs/dbt-core/issues/6241))
|
||||
- Clarify Error Message for how many models are allowed in a Python file ([#6245](https://github.com/dbt-labs/dbt-core/issues/6245))
|
||||
- Fix typo in util.py ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904))
|
||||
- After this, will be possible to use default values for dbt.config.get ([#6309](https://github.com/dbt-labs/dbt-core/issues/6309))
|
||||
- Use full path for writing manifest ([#6055](https://github.com/dbt-labs/dbt-core/issues/6055))
|
||||
- [CT-1284] Change Python model default materialization to table ([#6345](https://github.com/dbt-labs/dbt-core/issues/6345))
|
||||
- add pre-commit install to make dev script in Makefile ([#6269](https://github.com/dbt-labs/dbt-core/issues/6269))
|
||||
- Late-rendering for `pre_` and `post_hook`s in `dbt_project.yml` ([#6411](https://github.com/dbt-labs/dbt-core/issues/6411))
|
||||
- [CT-1284] Change Python model default materialization to table ([#5989](https://github.com/dbt-labs/dbt-core/issues/5989))
|
||||
- [CT-1591] Don't parse empty Python files ([#6345](https://github.com/dbt-labs/dbt-core/issues/6345))
|
||||
- Repair a regression which prevented basic logging before the logging subsystem is completely configured. ([#6434](https://github.com/dbt-labs/dbt-core/issues/6434))
|
||||
- fix docs generate --defer by adding defer_to_manifest to before_run ([#6488](https://github.com/dbt-labs/dbt-core/issues/6488))
|
||||
- Bug when partial parsing with an empty schema file ([#4850](https://github.com/dbt-labs/dbt-core/issues/4850))
|
||||
- Fix DBT_FAVOR_STATE env var ([#5859](https://github.com/dbt-labs/dbt-core/issues/5859))
|
||||
- Restore historical behavior of certain disabled test messages, so that they are at the less obtrusive debug level, rather than the warning level. ([#6501](https://github.com/dbt-labs/dbt-core/issues/6501))
|
||||
- Bump mashumuro version to get regression fix and add unit test to verify that fix. ([#6428](https://github.com/dbt-labs/dbt-core/issues/6428))
|
||||
- Call update_event_status earlier for node results. Rename event 'HookFinished' -> FinishedRunningStats ([#6571](https://github.com/dbt-labs/dbt-core/issues/6571))
|
||||
- Provide backward compatibility for `get_merge_sql` arguments ([#6625](https://github.com/dbt-labs/dbt-core/issues/6625))
|
||||
- Fix behavior of --favor-state with --defer ([#6617](https://github.com/dbt-labs/dbt-core/issues/6617))
|
||||
- Include adapter_response in NodeFinished run_result log event ([#6703](https://github.com/dbt-labs/dbt-core/issues/6703))
|
||||
|
||||
### Docs
|
||||
|
||||
@@ -41,6 +129,7 @@
|
||||
- ([dbt-docs/#5880](https://github.com/dbt-labs/dbt-docs/issues/5880))
|
||||
- Fix rendering of sample code for metrics ([dbt-docs/#323](https://github.com/dbt-labs/dbt-docs/issues/323))
|
||||
- Alphabetize `core/dbt/README.md` ([dbt-docs/#6368](https://github.com/dbt-labs/dbt-docs/issues/6368))
|
||||
- Updated minor typos encountered when skipping profile setup ([dbt-docs/#6529](https://github.com/dbt-labs/dbt-docs/issues/6529))
|
||||
|
||||
### Under the Hood
|
||||
|
||||
@@ -64,6 +153,14 @@
|
||||
- Fix intermittent database connection failure in Windows CI test ([#6394](https://github.com/dbt-labs/dbt-core/issues/6394))
|
||||
- Refactor and clean up manifest nodes ([#6426](https://github.com/dbt-labs/dbt-core/issues/6426))
|
||||
- Restore important legacy logging behaviors, following refactor which removed them ([#6437](https://github.com/dbt-labs/dbt-core/issues/6437))
|
||||
- Treat dense text blobs as binary for `git grep` ([#6294](https://github.com/dbt-labs/dbt-core/issues/6294))
|
||||
- Prune partial parsing logging events ([#6313](https://github.com/dbt-labs/dbt-core/issues/6313))
|
||||
- Updating the deprecation warning in the metric attributes renamed event ([#6507](https://github.com/dbt-labs/dbt-core/issues/6507))
|
||||
- [CT-1693] Port severity test to Pytest ([#6466](https://github.com/dbt-labs/dbt-core/issues/6466))
|
||||
- [CT-1694] Deprecate event tracking tests ([#6467](https://github.com/dbt-labs/dbt-core/issues/6467))
|
||||
- Reorganize structured logging events to have two top keys ([#6311](https://github.com/dbt-labs/dbt-core/issues/6311))
|
||||
- Combine some logging events ([#1716](https://github.com/dbt-labs/dbt-core/issues/1716), [#1717](https://github.com/dbt-labs/dbt-core/issues/1717), [#1719](https://github.com/dbt-labs/dbt-core/issues/1719))
|
||||
- Check length of escaped strings in the adapter test ([#6566](https://github.com/dbt-labs/dbt-core/issues/6566))
|
||||
|
||||
### Dependencies
|
||||
|
||||
@@ -71,29 +168,35 @@
|
||||
- Bump black from 22.8.0 to 22.10.0 ([#6019](https://github.com/dbt-labs/dbt-core/pull/6019))
|
||||
- Bump mashumaro[msgpack] from 3.0.4 to 3.1.1 in /core ([#6108](https://github.com/dbt-labs/dbt-core/pull/6108))
|
||||
- Update colorama requirement from <0.4.6,>=0.3.9 to >=0.3.9,<0.4.7 in /core ([#6144](https://github.com/dbt-labs/dbt-core/pull/6144))
|
||||
- Bump mashumaro[msgpack] from 3.1.1 to 3.2 in /core ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904))
|
||||
- Bump mashumaro[msgpack] from 3.1.1 to 3.2 in /core ([#6375](https://github.com/dbt-labs/dbt-core/pull/6375))
|
||||
- Update agate requirement from <1.6.4,>=1.6 to >=1.6,<1.7.1 in /core ([#6506](https://github.com/dbt-labs/dbt-core/pull/6506))
|
||||
|
||||
### Contributors
|
||||
- [@NiallRees](https://github.com/NiallRees) ([#5859](https://github.com/dbt-labs/dbt-core/issues/5859))
|
||||
- [@agpapa](https://github.com/agpapa) ([#6365](https://github.com/dbt-labs/dbt-core/issues/6365))
|
||||
- [@andy-clapson](https://github.com/andy-clapson) ([dbt-docs/#5791](https://github.com/dbt-labs/dbt-docs/issues/5791))
|
||||
- [@callum-mcdata](https://github.com/callum-mcdata) ([#6398](https://github.com/dbt-labs/dbt-core/issues/6398), [#6507](https://github.com/dbt-labs/dbt-core/issues/6507))
|
||||
- [@chamini2](https://github.com/chamini2) ([#6041](https://github.com/dbt-labs/dbt-core/issues/6041))
|
||||
- [@daniel-murray](https://github.com/daniel-murray) ([#2968](https://github.com/dbt-labs/dbt-core/issues/2968))
|
||||
- [@dave-connors-3](https://github.com/dave-connors-3) ([#5990](https://github.com/dbt-labs/dbt-core/issues/5990))
|
||||
- [@dbeatty10](https://github.com/dbeatty10) ([dbt-docs/#6368](https://github.com/dbt-labs/dbt-docs/issues/6368), [#6394](https://github.com/dbt-labs/dbt-core/issues/6394))
|
||||
- [@daniel-murray](https://github.com/daniel-murray) ([#5016](https://github.com/dbt-labs/dbt-core/issues/5016))
|
||||
- [@dave-connors-3](https://github.com/dave-connors-3) ([#5680](https://github.com/dbt-labs/dbt-core/issues/5680), [#5990](https://github.com/dbt-labs/dbt-core/issues/5990), [#6625](https://github.com/dbt-labs/dbt-core/issues/6625))
|
||||
- [@dbeatty10](https://github.com/dbeatty10) ([#6411](https://github.com/dbt-labs/dbt-core/issues/6411), [dbt-docs/#6368](https://github.com/dbt-labs/dbt-docs/issues/6368), [#6394](https://github.com/dbt-labs/dbt-core/issues/6394), [#6294](https://github.com/dbt-labs/dbt-core/issues/6294), [#6566](https://github.com/dbt-labs/dbt-core/issues/6566))
|
||||
- [@devmessias](https://github.com/devmessias) ([#6309](https://github.com/dbt-labs/dbt-core/issues/6309))
|
||||
- [@eltociear](https://github.com/eltociear) ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904))
|
||||
- [@eve-johns](https://github.com/eve-johns) ([#6068](https://github.com/dbt-labs/dbt-core/issues/6068))
|
||||
- [@haritamar](https://github.com/haritamar) ([#6246](https://github.com/dbt-labs/dbt-core/issues/6246))
|
||||
- [@jared-rimmer](https://github.com/jared-rimmer) ([#5486](https://github.com/dbt-labs/dbt-core/issues/5486))
|
||||
- [@josephberni](https://github.com/josephberni) ([#2968](https://github.com/dbt-labs/dbt-core/issues/2968))
|
||||
- [@josephberni](https://github.com/josephberni) ([#5016](https://github.com/dbt-labs/dbt-core/issues/5016))
|
||||
- [@joshuataylor](https://github.com/joshuataylor) ([#6147](https://github.com/dbt-labs/dbt-core/issues/6147))
|
||||
- [@justbldwn](https://github.com/justbldwn) ([#6245](https://github.com/dbt-labs/dbt-core/issues/6245))
|
||||
- [@justbldwn](https://github.com/justbldwn) ([#6241](https://github.com/dbt-labs/dbt-core/issues/6241), [#6245](https://github.com/dbt-labs/dbt-core/issues/6245), [#6269](https://github.com/dbt-labs/dbt-core/issues/6269))
|
||||
- [@luke-bassett](https://github.com/luke-bassett) ([#1350](https://github.com/dbt-labs/dbt-core/issues/1350))
|
||||
- [@max-sixty](https://github.com/max-sixty) ([#5946](https://github.com/dbt-labs/dbt-core/issues/5946), [#5983](https://github.com/dbt-labs/dbt-core/issues/5983), [#6028](https://github.com/dbt-labs/dbt-core/issues/6028))
|
||||
- [@mivanicova](https://github.com/mivanicova) ([#6488](https://github.com/dbt-labs/dbt-core/issues/6488))
|
||||
- [@nshuman1](https://github.com/nshuman1) ([dbt-docs/#6529](https://github.com/dbt-labs/dbt-docs/issues/6529))
|
||||
- [@paulbenschmidt](https://github.com/paulbenschmidt) ([dbt-docs/#5880](https://github.com/dbt-labs/dbt-docs/issues/5880))
|
||||
- [@pgoslatara](https://github.com/pgoslatara) ([#5929](https://github.com/dbt-labs/dbt-core/issues/5929))
|
||||
- [@racheldaniel](https://github.com/racheldaniel) ([#6201](https://github.com/dbt-labs/dbt-core/issues/6201))
|
||||
- [@timle2](https://github.com/timle2) ([#4205](https://github.com/dbt-labs/dbt-core/issues/4205))
|
||||
- [@dave-connors-3](https://github.com/dave-connors-3) ([#5680](https://github.com/dbt-labs/dbt-core/issues/5680))
|
||||
|
||||
- [@tmastny](https://github.com/tmastny) ([#6216](https://github.com/dbt-labs/dbt-core/issues/6216))
|
||||
|
||||
## Previous Releases
|
||||
|
||||
|
||||
@@ -90,10 +90,11 @@ def _generate_stats(manifest: Manifest):
|
||||
|
||||
def _add_prepended_cte(prepended_ctes, new_cte):
|
||||
for cte in prepended_ctes:
|
||||
if cte.id == new_cte.id:
|
||||
if cte.id == new_cte.id and new_cte.sql:
|
||||
cte.sql = new_cte.sql
|
||||
return
|
||||
prepended_ctes.append(new_cte)
|
||||
if new_cte.sql:
|
||||
prepended_ctes.append(new_cte)
|
||||
|
||||
|
||||
def _extend_prepended_ctes(prepended_ctes, new_prepended_ctes):
|
||||
@@ -257,16 +258,18 @@ class Compiler:
|
||||
inserting CTEs into the SQL.
|
||||
"""
|
||||
if model.compiled_code is None:
|
||||
raise DbtRuntimeError("Cannot inject ctes into an unparsed node", model)
|
||||
raise DbtRuntimeError("Cannot inject ctes into an uncompiled node", model)
|
||||
|
||||
# extra_ctes_injected flag says that we've already recursively injected the ctes
|
||||
if model.extra_ctes_injected:
|
||||
return (model, model.extra_ctes)
|
||||
|
||||
# Just to make it plain that nothing is actually injected for this case
|
||||
if not model.extra_ctes:
|
||||
if len(model.extra_ctes) == 0:
|
||||
# SeedNodes don't have compilation attributes
|
||||
if not isinstance(model, SeedNode):
|
||||
model.extra_ctes_injected = True
|
||||
manifest.update_node(model)
|
||||
return (model, model.extra_ctes)
|
||||
return (model, [])
|
||||
|
||||
# This stores the ctes which will all be recursively
|
||||
# gathered and then "injected" into the model.
|
||||
@@ -275,7 +278,8 @@ class Compiler:
|
||||
# extra_ctes are added to the model by
|
||||
# RuntimeRefResolver.create_relation, which adds an
|
||||
# extra_cte for every model relation which is an
|
||||
# ephemeral model.
|
||||
# ephemeral model. InjectedCTEs have a unique_id and sql.
|
||||
# extra_ctes start out with sql set to None, and the sql is set in this loop.
|
||||
for cte in model.extra_ctes:
|
||||
if cte.id not in manifest.nodes:
|
||||
raise DbtInternalError(
|
||||
@@ -288,23 +292,23 @@ class Compiler:
|
||||
if not cte_model.is_ephemeral_model:
|
||||
raise DbtInternalError(f"{cte.id} is not ephemeral")
|
||||
|
||||
# This model has already been compiled, so it's been
|
||||
# through here before
|
||||
if getattr(cte_model, "compiled", False):
|
||||
# This model has already been compiled and extra_ctes_injected, so it's been
|
||||
# through here before. We already checked above for extra_ctes_injected, but
|
||||
# checking again because updates maybe have happened in another thread.
|
||||
if cte_model.compiled is True and cte_model.extra_ctes_injected is True:
|
||||
new_prepended_ctes = cte_model.extra_ctes
|
||||
|
||||
# if the cte_model isn't compiled, i.e. first time here
|
||||
else:
|
||||
# This is an ephemeral parsed model that we can compile.
|
||||
# Compile and update the node
|
||||
cte_model = self._compile_node(cte_model, manifest, extra_context)
|
||||
# recursively call this method
|
||||
# Render the raw_code and set compiled to True
|
||||
cte_model = self._compile_code(cte_model, manifest, extra_context)
|
||||
# recursively call this method, sets extra_ctes_injected to True
|
||||
cte_model, new_prepended_ctes = self._recursively_prepend_ctes(
|
||||
cte_model, manifest, extra_context
|
||||
)
|
||||
# Save compiled SQL file and sync manifest
|
||||
# Write compiled SQL file
|
||||
self._write_node(cte_model)
|
||||
manifest.sync_update_node(cte_model)
|
||||
|
||||
_extend_prepended_ctes(prepended_ctes, new_prepended_ctes)
|
||||
|
||||
@@ -318,20 +322,21 @@ class Compiler:
|
||||
model.compiled_code,
|
||||
prepended_ctes,
|
||||
)
|
||||
model._pre_injected_sql = model.compiled_code
|
||||
model.compiled_code = injected_sql
|
||||
model.extra_ctes_injected = True
|
||||
model.extra_ctes = prepended_ctes
|
||||
model.validate(model.to_dict(omit_none=True))
|
||||
manifest.update_node(model)
|
||||
# Check again before updating for multi-threading
|
||||
if not model.extra_ctes_injected:
|
||||
model._pre_injected_sql = model.compiled_code
|
||||
model.compiled_code = injected_sql
|
||||
model.extra_ctes = prepended_ctes
|
||||
model.extra_ctes_injected = True
|
||||
|
||||
return model, prepended_ctes
|
||||
# if model.extra_ctes is not set to prepended ctes, something went wrong
|
||||
return model, model.extra_ctes
|
||||
|
||||
# Sets compiled fields in the ManifestSQLNode passed in,
|
||||
# Sets compiled_code and compiled flag in the ManifestSQLNode passed in,
|
||||
# creates a "context" dictionary for jinja rendering,
|
||||
# and then renders the "compiled_code" using the node, the
|
||||
# raw_code and the context.
|
||||
def _compile_node(
|
||||
def _compile_code(
|
||||
self,
|
||||
node: ManifestSQLNode,
|
||||
manifest: Manifest,
|
||||
@@ -340,16 +345,6 @@ class Compiler:
|
||||
if extra_context is None:
|
||||
extra_context = {}
|
||||
|
||||
data = node.to_dict(omit_none=True)
|
||||
data.update(
|
||||
{
|
||||
"compiled": False,
|
||||
"compiled_code": None,
|
||||
"extra_ctes_injected": False,
|
||||
"extra_ctes": [],
|
||||
}
|
||||
)
|
||||
|
||||
if node.language == ModelLanguage.python:
|
||||
# TODO could we also 'minify' this code at all? just aesthetic, not functional
|
||||
|
||||
@@ -380,6 +375,18 @@ class Compiler:
|
||||
|
||||
node.compiled = True
|
||||
|
||||
# relation_name is set at parse time, except for tests without store_failures,
|
||||
# but cli param can turn on store_failures, so we set here.
|
||||
if (
|
||||
node.resource_type == NodeType.Test
|
||||
and node.relation_name is None
|
||||
and node.is_relational
|
||||
):
|
||||
adapter = get_adapter(self.config)
|
||||
relation_cls = adapter.Relation
|
||||
relation_name = str(relation_cls.create_from(self.config, node))
|
||||
node.relation_name = relation_name
|
||||
|
||||
return node
|
||||
|
||||
def write_graph_file(self, linker: Linker, manifest: Manifest):
|
||||
@@ -510,11 +517,11 @@ class Compiler:
|
||||
) -> ManifestSQLNode:
|
||||
"""This is the main entry point into this code. It's called by
|
||||
CompileRunner.compile, GenericRPCRunner.compile, and
|
||||
RunTask.get_hook_sql. It calls '_compile_node' to convert
|
||||
the node into a compiled node, and then calls the
|
||||
RunTask.get_hook_sql. It calls '_compile_code' to render
|
||||
the node's raw_code into compiled_code, and then calls the
|
||||
recursive method to "prepend" the ctes.
|
||||
"""
|
||||
node = self._compile_node(node, manifest, extra_context)
|
||||
node = self._compile_code(node, manifest, extra_context)
|
||||
|
||||
node, _ = self._recursively_prepend_ctes(node, manifest, extra_context)
|
||||
if write:
|
||||
|
||||
@@ -12,7 +12,6 @@ from typing import (
|
||||
)
|
||||
from typing_extensions import Protocol, runtime_checkable
|
||||
|
||||
import hashlib
|
||||
import os
|
||||
|
||||
from dbt import flags, deprecations
|
||||
@@ -30,7 +29,7 @@ from dbt.graph import SelectionSpec
|
||||
from dbt.helper_types import NoValue
|
||||
from dbt.semver import VersionSpecifier, versions_compatible
|
||||
from dbt.version import get_installed_version
|
||||
from dbt.utils import MultiDict
|
||||
from dbt.utils import MultiDict, md5
|
||||
from dbt.node_types import NodeType
|
||||
from dbt.config.selectors import SelectorDict
|
||||
from dbt.contracts.project import (
|
||||
@@ -663,7 +662,7 @@ class Project:
|
||||
return partial.render(renderer)
|
||||
|
||||
def hashed_name(self):
|
||||
return hashlib.md5(self.project_name.encode("utf-8")).hexdigest()
|
||||
return md5(self.project_name)
|
||||
|
||||
def get_selector(self, name: str) -> Union[SelectionSpec, bool]:
|
||||
if name not in self.selectors:
|
||||
|
||||
@@ -9,7 +9,7 @@ from dbt.config import Profile, Project, read_user_config
|
||||
from dbt.config.renderer import DbtProjectYamlRenderer, ProfileRenderer
|
||||
from dbt.events.functions import fire_event
|
||||
from dbt.events.types import InvalidOptionYAML
|
||||
from dbt.exceptions import DbtValidationError, OptionNotYamlDict
|
||||
from dbt.exceptions import DbtValidationError, OptionNotYamlDictError
|
||||
|
||||
|
||||
def parse_cli_vars(var_string: str) -> Dict[str, Any]:
|
||||
@@ -23,7 +23,7 @@ def parse_cli_yaml_string(var_string: str, cli_option_name: str) -> Dict[str, An
|
||||
if var_type is dict:
|
||||
return cli_vars
|
||||
else:
|
||||
raise OptionNotYamlDict(var_type, cli_option_name)
|
||||
raise OptionNotYamlDictError(var_type, cli_option_name)
|
||||
except DbtValidationError:
|
||||
fire_event(InvalidOptionYAML(option_name=cli_option_name))
|
||||
raise
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
import abc
|
||||
import itertools
|
||||
import hashlib
|
||||
from dataclasses import dataclass, field
|
||||
from typing import (
|
||||
Any,
|
||||
@@ -13,7 +12,7 @@ from typing import (
|
||||
Callable,
|
||||
)
|
||||
from dbt.exceptions import DbtInternalError
|
||||
from dbt.utils import translate_aliases
|
||||
from dbt.utils import translate_aliases, md5
|
||||
from dbt.events.functions import fire_event
|
||||
from dbt.events.types import NewConnectionOpening
|
||||
from dbt.events.contextvars import get_node_info
|
||||
@@ -142,7 +141,7 @@ class Credentials(ExtensibleDbtClassMixin, Replaceable, metaclass=abc.ABCMeta):
|
||||
raise NotImplementedError("unique_field not implemented for base credentials class")
|
||||
|
||||
def hashed_unique_field(self) -> str:
|
||||
return hashlib.md5(self.unique_field.encode("utf-8")).hexdigest()
|
||||
return md5(self.unique_field)
|
||||
|
||||
def connection_info(self, *, with_aliases: bool = False) -> Iterable[Tuple[str, Any]]:
|
||||
"""Return an ordered iterator of key/value pairs for pretty-printing."""
|
||||
|
||||
@@ -646,24 +646,6 @@ class Manifest(MacroMethods, DataClassMessagePackMixin, dbtClassMixin):
|
||||
obj._lock = flags.MP_CONTEXT.Lock()
|
||||
return obj
|
||||
|
||||
def sync_update_node(self, new_node: ManifestNode) -> ManifestNode:
|
||||
"""update the node with a lock. The only time we should want to lock is
|
||||
when compiling an ephemeral ancestor of a node at runtime, because
|
||||
multiple threads could be just-in-time compiling the same ephemeral
|
||||
dependency, and we want them to have a consistent view of the manifest.
|
||||
|
||||
If the existing node is not compiled, update it with the new node and
|
||||
return that. If the existing node is compiled, do not update the
|
||||
manifest and return the existing node.
|
||||
"""
|
||||
with self._lock:
|
||||
existing = self.nodes[new_node.unique_id]
|
||||
if getattr(existing, "compiled", False):
|
||||
# already compiled
|
||||
return existing
|
||||
_update_into(self.nodes, new_node)
|
||||
return new_node
|
||||
|
||||
def update_exposure(self, new_exposure: Exposure):
|
||||
_update_into(self.exposures, new_exposure)
|
||||
|
||||
@@ -1181,7 +1163,7 @@ class WritableManifest(ArtifactMixin):
|
||||
selectors: Mapping[UniqueID, Any] = field(
|
||||
metadata=dict(description=("The selectors defined in selectors.yml"))
|
||||
)
|
||||
disabled: Optional[Mapping[UniqueID, List[ResultNode]]] = field(
|
||||
disabled: Optional[Mapping[UniqueID, List[GraphMemberNode]]] = field(
|
||||
metadata=dict(description="A mapping of the disabled nodes in the target")
|
||||
)
|
||||
parent_map: Optional[NodeEdgeMap] = field(
|
||||
|
||||
@@ -37,6 +37,7 @@ from dbt.contracts.graph.unparsed import (
|
||||
from dbt.contracts.util import Replaceable, AdditionalPropertiesMixin
|
||||
from dbt.events.proto_types import NodeInfo
|
||||
from dbt.events.functions import warn_or_error
|
||||
from dbt.exceptions import ParsingError
|
||||
from dbt.events.types import (
|
||||
SeedIncreased,
|
||||
SeedExceedsLimitSamePath,
|
||||
@@ -409,8 +410,10 @@ class CompiledNode(ParsedNode):
|
||||
do if extra_ctes were an OrderedDict
|
||||
"""
|
||||
for cte in self.extra_ctes:
|
||||
# Because it's possible that multiple threads are compiling the
|
||||
# node at the same time, we don't want to overwrite already compiled
|
||||
# sql in the extra_ctes with empty sql.
|
||||
if cte.id == cte_id:
|
||||
cte.sql = sql
|
||||
break
|
||||
else:
|
||||
self.extra_ctes.append(InjectedCTE(id=cte_id, sql=sql))
|
||||
@@ -482,6 +485,7 @@ class SeedNode(ParsedNode): # No SQLDefaults!
|
||||
# seeds need the root_path because the contents are not loaded initially
|
||||
# and we need the root_path to load the seed later
|
||||
root_path: Optional[str] = None
|
||||
depends_on: MacroDependsOn = field(default_factory=MacroDependsOn)
|
||||
|
||||
def same_seeds(self, other: "SeedNode") -> bool:
|
||||
# for seeds, we check the hashes. If the hashes are different types,
|
||||
@@ -523,6 +527,39 @@ class SeedNode(ParsedNode): # No SQLDefaults!
|
||||
"""Seeds are never empty"""
|
||||
return False
|
||||
|
||||
def _disallow_implicit_dependencies(self):
|
||||
"""Disallow seeds to take implicit upstream dependencies via pre/post hooks"""
|
||||
# Seeds are root nodes in the DAG. They cannot depend on other nodes.
|
||||
# However, it's possible to define pre- and post-hooks on seeds, and for those
|
||||
# hooks to include {{ ref(...) }}. This worked in previous versions, but it
|
||||
# was never officially documented or supported behavior. Let's raise an explicit error,
|
||||
# which will surface during parsing if the user has written code such that we attempt
|
||||
# to capture & record a ref/source/metric call on the SeedNode.
|
||||
# For more details: https://github.com/dbt-labs/dbt-core/issues/6806
|
||||
hooks = [f'- pre_hook: "{hook.sql}"' for hook in self.config.pre_hook] + [
|
||||
f'- post_hook: "{hook.sql}"' for hook in self.config.post_hook
|
||||
]
|
||||
hook_list = "\n".join(hooks)
|
||||
message = f"""
|
||||
Seeds cannot depend on other nodes. dbt detected a seed with a pre- or post-hook
|
||||
that calls 'ref', 'source', or 'metric', either directly or indirectly via other macros.
|
||||
|
||||
Error raised for '{self.unique_id}', which has these hooks defined: \n{hook_list}
|
||||
"""
|
||||
raise ParsingError(message)
|
||||
|
||||
@property
|
||||
def refs(self):
|
||||
self._disallow_implicit_dependencies()
|
||||
|
||||
@property
|
||||
def sources(self):
|
||||
self._disallow_implicit_dependencies()
|
||||
|
||||
@property
|
||||
def metrics(self):
|
||||
self._disallow_implicit_dependencies()
|
||||
|
||||
def same_body(self, other) -> bool:
|
||||
return self.same_seeds(other)
|
||||
|
||||
@@ -531,8 +568,8 @@ class SeedNode(ParsedNode): # No SQLDefaults!
|
||||
return []
|
||||
|
||||
@property
|
||||
def depends_on_macros(self):
|
||||
return []
|
||||
def depends_on_macros(self) -> List[str]:
|
||||
return self.depends_on.macros
|
||||
|
||||
@property
|
||||
def extra_ctes(self):
|
||||
|
||||
@@ -13,7 +13,7 @@ from dbt.events.types import TimingInfoCollected
|
||||
from dbt.events.proto_types import RunResultMsg, TimingInfoMsg
|
||||
from dbt.events.contextvars import get_node_info
|
||||
from dbt.logger import TimingProcessor
|
||||
from dbt.utils import lowercase, cast_to_str, cast_to_int
|
||||
from dbt.utils import lowercase, cast_to_str, cast_to_int, cast_dict_to_dict_of_strings
|
||||
from dbt.dataclass_schema import dbtClassMixin, StrEnum
|
||||
|
||||
import agate
|
||||
@@ -130,7 +130,6 @@ class BaseResult(dbtClassMixin):
|
||||
return data
|
||||
|
||||
def to_msg(self):
|
||||
# TODO: add more fields
|
||||
msg = RunResultMsg()
|
||||
msg.status = str(self.status)
|
||||
msg.message = cast_to_str(self.message)
|
||||
@@ -138,7 +137,7 @@ class BaseResult(dbtClassMixin):
|
||||
msg.execution_time = self.execution_time
|
||||
msg.num_failures = cast_to_int(self.failures)
|
||||
msg.timing_info = [ti.to_msg() for ti in self.timing]
|
||||
# adapter_response
|
||||
msg.adapter_response = cast_dict_to_dict_of_strings(self.adapter_response)
|
||||
return msg
|
||||
|
||||
|
||||
|
||||
@@ -250,7 +250,6 @@ def upgrade_seed_content(node_content):
|
||||
"refs",
|
||||
"sources",
|
||||
"metrics",
|
||||
"depends_on",
|
||||
"compiled_path",
|
||||
"compiled",
|
||||
"compiled_code",
|
||||
@@ -260,6 +259,8 @@ def upgrade_seed_content(node_content):
|
||||
):
|
||||
if attr_name in node_content:
|
||||
del node_content[attr_name]
|
||||
# In v1.4, we switched SeedNode.depends_on from DependsOn to MacroDependsOn
|
||||
node_content.get("depends_on", {}).pop("nodes", None)
|
||||
|
||||
|
||||
def upgrade_manifest_json(manifest: dict) -> dict:
|
||||
@@ -283,7 +284,7 @@ def upgrade_manifest_json(manifest: dict) -> dict:
|
||||
if "root_path" in exposure_content:
|
||||
del exposure_content["root_path"]
|
||||
for source_content in manifest.get("sources", {}).values():
|
||||
if "root_path" in exposure_content:
|
||||
if "root_path" in source_content:
|
||||
del source_content["root_path"]
|
||||
for macro_content in manifest.get("macros", {}).values():
|
||||
if "root_path" in macro_content:
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import os
|
||||
import hashlib
|
||||
from typing import List, Optional
|
||||
|
||||
from dbt.clients import git, system
|
||||
@@ -12,10 +11,11 @@ from dbt.deps.base import PinnedPackage, UnpinnedPackage, get_downloads_path
|
||||
from dbt.exceptions import ExecutableError, MultipleVersionGitDepsError
|
||||
from dbt.events.functions import fire_event, warn_or_error
|
||||
from dbt.events.types import EnsureGitInstalled, DepsUnpinned
|
||||
from dbt.utils import md5
|
||||
|
||||
|
||||
def md5sum(s: str):
|
||||
return hashlib.md5(s.encode("latin-1")).hexdigest()
|
||||
return md5(s, "latin-1")
|
||||
|
||||
|
||||
class GitPackageMixin:
|
||||
|
||||
@@ -88,41 +88,40 @@ class _Logger:
|
||||
self.level: EventLevel = config.level
|
||||
self.event_manager: EventManager = event_manager
|
||||
self._python_logger: Optional[logging.Logger] = config.logger
|
||||
self._stream: Optional[TextIO] = config.output_stream
|
||||
|
||||
if config.output_stream is not None:
|
||||
stream_handler = logging.StreamHandler(config.output_stream)
|
||||
self._python_logger = self._get_python_log_for_handler(stream_handler)
|
||||
|
||||
if config.output_file_name:
|
||||
log = logging.getLogger(config.name)
|
||||
log.setLevel(_log_level_map[config.level])
|
||||
handler = RotatingFileHandler(
|
||||
file_handler = RotatingFileHandler(
|
||||
filename=str(config.output_file_name),
|
||||
encoding="utf8",
|
||||
maxBytes=10 * 1024 * 1024, # 10 mb
|
||||
backupCount=5,
|
||||
)
|
||||
self._python_logger = self._get_python_log_for_handler(file_handler)
|
||||
|
||||
handler.setFormatter(logging.Formatter(fmt="%(message)s"))
|
||||
log.handlers.clear()
|
||||
log.addHandler(handler)
|
||||
|
||||
self._python_logger = log
|
||||
def _get_python_log_for_handler(self, handler: logging.Handler):
|
||||
log = logging.getLogger(self.name)
|
||||
log.setLevel(_log_level_map[self.level])
|
||||
handler.setFormatter(logging.Formatter(fmt="%(message)s"))
|
||||
log.handlers.clear()
|
||||
log.addHandler(handler)
|
||||
return log
|
||||
|
||||
def create_line(self, msg: EventMsg) -> str:
|
||||
raise NotImplementedError()
|
||||
|
||||
def write_line(self, msg: EventMsg):
|
||||
line = self.create_line(msg)
|
||||
python_level = _log_level_map[EventLevel(msg.info.level)]
|
||||
if self._python_logger is not None:
|
||||
send_to_logger(self._python_logger, msg.info.level, line)
|
||||
elif self._stream is not None and _log_level_map[self.level] <= python_level:
|
||||
self._stream.write(line + "\n")
|
||||
|
||||
def flush(self):
|
||||
if self._python_logger is not None:
|
||||
for handler in self._python_logger.handlers:
|
||||
handler.flush()
|
||||
elif self._stream is not None:
|
||||
self._stream.flush()
|
||||
|
||||
|
||||
class _TextLogger(_Logger):
|
||||
|
||||
@@ -18,6 +18,14 @@ LOG_VERSION = 3
|
||||
metadata_vars: Optional[Dict[str, str]] = None
|
||||
|
||||
|
||||
# The "fallback" logger is used as a stop-gap so that console logging works before the logging
|
||||
# configuration is fully loaded.
|
||||
def setup_fallback_logger(use_legacy: bool, level: EventLevel) -> None:
|
||||
cleanup_event_logger()
|
||||
config = _get_logbook_log_config(level) if use_legacy else _get_stdout_config(level)
|
||||
EVENT_MANAGER.add_logger(config)
|
||||
|
||||
|
||||
def setup_event_logger(log_path: str, level_override: Optional[EventLevel] = None):
|
||||
cleanup_event_logger()
|
||||
make_log_dir_if_missing(log_path)
|
||||
@@ -94,6 +102,7 @@ def _get_logbook_log_config(level: Optional[EventLevel] = None) -> LoggerConfig:
|
||||
config.name = "logbook_log"
|
||||
config.filter = NoFilter if flags.LOG_CACHE_EVENTS else lambda e: not isinstance(e.data, Cache)
|
||||
config.logger = GLOBAL_LOGGER
|
||||
config.output_stream = None
|
||||
return config
|
||||
|
||||
|
||||
@@ -113,9 +122,7 @@ def cleanup_event_logger():
|
||||
# currently fire before logs can be configured by setup_event_logger(), we
|
||||
# create a default configuration with default settings and no file output.
|
||||
EVENT_MANAGER: EventManager = EventManager()
|
||||
EVENT_MANAGER.add_logger(
|
||||
_get_logbook_log_config() if flags.ENABLE_LEGACY_LOGGER else _get_stdout_config()
|
||||
)
|
||||
setup_fallback_logger(bool(flags.ENABLE_LEGACY_LOGGER), EventLevel.INFO)
|
||||
|
||||
|
||||
# This global, and the following two functions for capturing stdout logs are
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user