forked from repo-mirrors/dbt-core
Compare commits
65 Commits
test_mater
...
remove-ipd
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
83c40d7fb5 | ||
|
|
6bfba6d116 | ||
|
|
d23d414dfe | ||
|
|
6e7e572e86 | ||
|
|
d57542307b | ||
|
|
7f2e9f94c0 | ||
|
|
27b2e9ac54 | ||
|
|
a6890609e0 | ||
|
|
3c30f96ab2 | ||
|
|
6fff2888d4 | ||
|
|
3f17044383 | ||
|
|
e1345d87cd | ||
|
|
6e9ff280e2 | ||
|
|
6defc86ef7 | ||
|
|
3fbb0a38ea | ||
|
|
d71385bdb6 | ||
|
|
c1925c67c2 | ||
|
|
54d186583b | ||
|
|
9026a0598c | ||
|
|
2d1215ed47 | ||
|
|
7f63c3d083 | ||
|
|
19c48e285e | ||
|
|
65f40b317e | ||
|
|
c94c891f73 | ||
|
|
fc45a51582 | ||
|
|
6042469c71 | ||
|
|
2584465169 | ||
|
|
9a81a4dfe3 | ||
|
|
b6a82446e5 | ||
|
|
17aca39e1c | ||
|
|
59744f18bb | ||
|
|
f1326f526c | ||
|
|
834ac716fd | ||
|
|
0487b96098 | ||
|
|
dbd36f06e4 | ||
|
|
38ada8a68e | ||
|
|
e58edaab2d | ||
|
|
c202e005cd | ||
|
|
8129862b3c | ||
|
|
4e8aa007cf | ||
|
|
fe88bfabbf | ||
|
|
5328a64df2 | ||
|
|
87c9974be1 | ||
|
|
f3f509da92 | ||
|
|
5e8dcec2c5 | ||
|
|
56783446db | ||
|
|
207cc0383d | ||
|
|
49ecd6a6a4 | ||
|
|
c109f39d82 | ||
|
|
fd778dceb5 | ||
|
|
e402241e0e | ||
|
|
a6c37c948d | ||
|
|
fd886cb7dd | ||
|
|
b089a471b7 | ||
|
|
ae294b643b | ||
|
|
0bd6df0d1b | ||
|
|
7b1d61c956 | ||
|
|
646a0c704f | ||
|
|
bbf4fc30a5 | ||
|
|
6baaa2bcb0 | ||
|
|
13a595722a | ||
|
|
3680b6ad0e | ||
|
|
4c29d48d1c | ||
|
|
e00eb9aa3a | ||
|
|
f5a94fc774 |
@@ -1,13 +1,15 @@
|
||||
[bumpversion]
|
||||
current_version = 1.3.0b2
|
||||
parse = (?P<major>\d+)
|
||||
\.(?P<minor>\d+)
|
||||
\.(?P<patch>\d+)
|
||||
((?P<prekind>a|b|rc)
|
||||
(?P<pre>\d+) # pre-release version num
|
||||
current_version = 1.3.4
|
||||
parse = (?P<major>[\d]+) # major version number
|
||||
\.(?P<minor>[\d]+) # minor version number
|
||||
\.(?P<patch>[\d]+) # patch version number
|
||||
(((?P<prekind>a|b|rc) # optional pre-release type
|
||||
?(?P<num>[\d]+?)) # optional pre-release version number
|
||||
\.?(?P<nightly>[a-z0-9]+\+[a-z]+)? # optional nightly release indicator
|
||||
)?
|
||||
serialize =
|
||||
{major}.{minor}.{patch}{prekind}{pre}
|
||||
{major}.{minor}.{patch}{prekind}{num}.{nightly}
|
||||
{major}.{minor}.{patch}{prekind}{num}
|
||||
{major}.{minor}.{patch}
|
||||
commit = False
|
||||
tag = False
|
||||
@@ -21,9 +23,11 @@ values =
|
||||
rc
|
||||
final
|
||||
|
||||
[bumpversion:part:pre]
|
||||
[bumpversion:part:num]
|
||||
first_value = 1
|
||||
|
||||
[bumpversion:part:nightly]
|
||||
|
||||
[bumpversion:file:core/setup.py]
|
||||
|
||||
[bumpversion:file:core/dbt/version.py]
|
||||
|
||||
@@ -1,38 +0,0 @@
|
||||
## dbt-core 1.3.0-b1 - July 29, 2022
|
||||
### Features
|
||||
- Python model inital version ([#5261](https://github.com/dbt-labs/dbt-core/issues/5261), [#5421](https://github.com/dbt-labs/dbt-core/pull/5421))
|
||||
- allows user to include the file extension for .py models in the dbt run -m command. ([#5289](https://github.com/dbt-labs/dbt-core/issues/5289), [#5295](https://github.com/dbt-labs/dbt-core/pull/5295))
|
||||
- Incremental materialization refactor and cleanup ([#5245](https://github.com/dbt-labs/dbt-core/issues/5245), [#5359](https://github.com/dbt-labs/dbt-core/pull/5359))
|
||||
- Python models can support incremental logic ([#0](https://github.com/dbt-labs/dbt-core/issues/0), [#35](https://github.com/dbt-labs/dbt-core/pull/35))
|
||||
- Add reusable function for retrying adapter connections. Utilize said function to add retries for Postgres (and Redshift). ([#5022](https://github.com/dbt-labs/dbt-core/issues/5022), [#5432](https://github.com/dbt-labs/dbt-core/pull/5432))
|
||||
- add exponential backoff to connection retries on Postgres (and Redshift) ([#5502](https://github.com/dbt-labs/dbt-core/issues/5502), [#5503](https://github.com/dbt-labs/dbt-core/pull/5503))
|
||||
### Fixes
|
||||
- Add context to compilation errors generated while rendering generic test configuration values. ([#5294](https://github.com/dbt-labs/dbt-core/issues/5294), [#5393](https://github.com/dbt-labs/dbt-core/pull/5393))
|
||||
- Rename try to strict for more intuitiveness ([#5475](https://github.com/dbt-labs/dbt-core/issues/5475), [#5477](https://github.com/dbt-labs/dbt-core/pull/5477))
|
||||
- Ignore empty strings passed in as secrets ([#5312](https://github.com/dbt-labs/dbt-core/issues/5312), [#5518](https://github.com/dbt-labs/dbt-core/pull/5518))
|
||||
- Fix handling of top-level exceptions ([#5564](https://github.com/dbt-labs/dbt-core/issues/5564), [#5560](https://github.com/dbt-labs/dbt-core/pull/5560))
|
||||
### Docs
|
||||
- Update dependency inline-source from ^6.1.5 to ^7.2.0 ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
|
||||
- Update dependency jest from ^26.2.2 to ^28.1.3 ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
|
||||
- Update dependency underscore from ^1.9.0 to ^1.13.4 ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
|
||||
- Update dependency webpack-cli from ^3.3.12 to ^4.7.0 ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
|
||||
- Update dependency webpack-dev-server from ^3.1.11 to ^4.9.3 ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
|
||||
- Searches no longer require perfect matches, and instead consider each word individually. `my model` or `model my` will now find `my_model`, without the need for underscores ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
|
||||
- Support the renaming of SQL to code happening in dbt-core ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
|
||||
### Under the Hood
|
||||
- Added language to tracked fields in run_model event ([#5571](https://github.com/dbt-labs/dbt-core/issues/5571), [#5469](https://github.com/dbt-labs/dbt-core/pull/5469))
|
||||
- Update mashumaro to 3.0.3 ([#4940](https://github.com/dbt-labs/dbt-core/issues/4940), [#5118](https://github.com/dbt-labs/dbt-core/pull/5118))
|
||||
- Add python incremental materialization test ([#0000](https://github.com/dbt-labs/dbt-core/issues/0000), [#5571](https://github.com/dbt-labs/dbt-core/pull/5571))
|
||||
### Dependencies
|
||||
- Upgrade to Jinja2==3.1.2 from Jinja2==2.11.3 ([#4748](https://github.com/dbt-labs/dbt-core/issues/4748), [#5465](https://github.com/dbt-labs/dbt-core/pull/5465))
|
||||
- Bump mypy from 0.961 to 0.971 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5495](https://github.com/dbt-labs/dbt-core/pull/5495))
|
||||
- Remove pin for MarkUpSafe from >=0.23,<2.1 ([#5506](https://github.com/dbt-labs/dbt-core/issues/5506), [#5507](https://github.com/dbt-labs/dbt-core/pull/5507))
|
||||
|
||||
### Contributors
|
||||
- [@Goodkat](https://github.com/Goodkat) ([#5518](https://github.com/dbt-labs/dbt-core/pull/5518))
|
||||
- [@drewbanin](https://github.com/drewbanin) ([#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
|
||||
- [@jeremyyeo](https://github.com/jeremyyeo) ([#5477](https://github.com/dbt-labs/dbt-core/pull/5477))
|
||||
- [@joellabes](https://github.com/joellabes) ([#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
|
||||
- [@leoebfolsom](https://github.com/leoebfolsom) ([#5295](https://github.com/dbt-labs/dbt-core/pull/5295))
|
||||
- [@nicholasyager](https://github.com/nicholasyager) ([#5393](https://github.com/dbt-labs/dbt-core/pull/5393))
|
||||
- [@tomasfarias](https://github.com/tomasfarias) ([#5432](https://github.com/dbt-labs/dbt-core/pull/5432))
|
||||
@@ -1,49 +0,0 @@
|
||||
## dbt-core 1.3.0-b2 - August 29, 2022
|
||||
|
||||
### Features
|
||||
|
||||
- Add `--defer` flag to dbt compile & dbt docs generate ([#4110](https://github.com/dbt-labs/dbt-core/issues/4110), [#4514](https://github.com/dbt-labs/dbt-core/pull/4514))
|
||||
- use MethodName.File when value ends with .csv ([#5578](https://github.com/dbt-labs/dbt-core/issues/5578), [#5581](https://github.com/dbt-labs/dbt-core/pull/5581))
|
||||
- Make `docs` configurable in `dbt_project.yml` and add a `node_color` attribute to change the color of nodes in the DAG ([#5333](https://github.com/dbt-labs/dbt-core/issues/5333), [#5397](https://github.com/dbt-labs/dbt-core/pull/5397))
|
||||
- Adding ResolvedMetricReference helper functions and tests ([#5567](https://github.com/dbt-labs/dbt-core/issues/5567), [#5607](https://github.com/dbt-labs/dbt-core/pull/5607))
|
||||
- Check dbt-core version requirements when installing Hub packages ([#5648](https://github.com/dbt-labs/dbt-core/issues/5648), [#5651](https://github.com/dbt-labs/dbt-core/pull/5651))
|
||||
|
||||
### Fixes
|
||||
|
||||
- Remove the default 256 characters limit on postgres character varying type when no limitation is set ([#5238](https://github.com/dbt-labs/dbt-core/issues/5238), [#5292](https://github.com/dbt-labs/dbt-core/pull/5292))
|
||||
- Include schema file config in unrendered_config ([#5338](https://github.com/dbt-labs/dbt-core/issues/5338), [#5344](https://github.com/dbt-labs/dbt-core/pull/5344))
|
||||
- Resolves #5351 - Do not consider shorter varchar cols as schema changes ([#5351](https://github.com/dbt-labs/dbt-core/issues/5351), [#5395](https://github.com/dbt-labs/dbt-core/pull/5395))
|
||||
- on_shchma_change fail verbosity enhancement ([#5504](https://github.com/dbt-labs/dbt-core/issues/5504), [#5505](https://github.com/dbt-labs/dbt-core/pull/5505))
|
||||
- Fix error rendering docs block in metrics description ([#5585](https://github.com/dbt-labs/dbt-core/issues/5585), [#5603](https://github.com/dbt-labs/dbt-core/pull/5603))
|
||||
- Extended validations for the project names ([#5379](https://github.com/dbt-labs/dbt-core/issues/5379), [#5620](https://github.com/dbt-labs/dbt-core/pull/5620))
|
||||
- Use sys.exit instead of exit ([#5621](https://github.com/dbt-labs/dbt-core/issues/5621), [#5627](https://github.com/dbt-labs/dbt-core/pull/5627))
|
||||
- Finishing logic upgrade to Redshift for name truncation collisions. ([#5586](https://github.com/dbt-labs/dbt-core/issues/5586), [#5656](https://github.com/dbt-labs/dbt-core/pull/5656))
|
||||
- multiple args for ref and source ([#5634](https://github.com/dbt-labs/dbt-core/issues/5634), [#5635](https://github.com/dbt-labs/dbt-core/pull/5635))
|
||||
- Fix Unexpected behavior when chaining methods on dbt-ref'ed/sourced dataframes ([#5646](https://github.com/dbt-labs/dbt-core/issues/5646), [#5677](https://github.com/dbt-labs/dbt-core/pull/5677))
|
||||
|
||||
### Docs
|
||||
|
||||
- Leverages `docs.node_color` from `dbt-core` to color nodes in the DAG ([dbt-docs/#44](https://github.com/dbt-labs/dbt-docs/issues/44), [dbt-docs/#281](https://github.com/dbt-labs/dbt-docs/pull/281))
|
||||
|
||||
### Under the Hood
|
||||
|
||||
- Save use of default env vars to manifest to enable partial parsing in those cases. ([#5155](https://github.com/dbt-labs/dbt-core/issues/5155), [#5589](https://github.com/dbt-labs/dbt-core/pull/5589))
|
||||
- add more information to log line interop test failures ([#5658](https://github.com/dbt-labs/dbt-core/issues/5658), [#5659](https://github.com/dbt-labs/dbt-core/pull/5659))
|
||||
- Add supported languages to materializations ([#5569](https://github.com/dbt-labs/dbt-core/issues/5569), [#5695](https://github.com/dbt-labs/dbt-core/pull/5695))
|
||||
|
||||
### Dependency
|
||||
|
||||
- Bump python from 3.10.5-slim-bullseye to 3.10.6-slim-bullseye in /docker ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5623](https://github.com/dbt-labs/dbt-core/pull/5623))
|
||||
- Bump mashumaro[msgpack] from 3.0.3 to 3.0.4 in /core ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5649](https://github.com/dbt-labs/dbt-core/pull/5649))
|
||||
|
||||
### Contributors
|
||||
- [@Goodkat](https://github.com/Goodkat) ([#5581](https://github.com/dbt-labs/dbt-core/pull/5581), [#5620](https://github.com/dbt-labs/dbt-core/pull/5620))
|
||||
- [@Ilanbenb](https://github.com/Ilanbenb) ([#5505](https://github.com/dbt-labs/dbt-core/pull/5505))
|
||||
- [@b-per](https://github.com/b-per) ([#5397](https://github.com/dbt-labs/dbt-core/pull/5397), [dbt-docs/#281](https://github.com/dbt-labs/dbt-docs/pull/281))
|
||||
- [@callum-mcdata](https://github.com/callum-mcdata) ([#5607](https://github.com/dbt-labs/dbt-core/pull/5607))
|
||||
- [@epapineau](https://github.com/epapineau) ([#5395](https://github.com/dbt-labs/dbt-core/pull/5395))
|
||||
- [@kadero](https://github.com/kadero) ([#4514](https://github.com/dbt-labs/dbt-core/pull/4514))
|
||||
- [@matt-winkler](https://github.com/matt-winkler) ([#5397](https://github.com/dbt-labs/dbt-core/pull/5397), [dbt-docs/#281](https://github.com/dbt-labs/dbt-docs/pull/281))
|
||||
- [@shrodingers](https://github.com/shrodingers) ([#5292](https://github.com/dbt-labs/dbt-core/pull/5292))
|
||||
- [@sungchun12](https://github.com/sungchun12) ([#5397](https://github.com/dbt-labs/dbt-core/pull/5397), [dbt-docs/#281](https://github.com/dbt-labs/dbt-docs/pull/281))
|
||||
- [@varun-dc](https://github.com/varun-dc) ([#5627](https://github.com/dbt-labs/dbt-core/pull/5627))
|
||||
140
.changes/1.3.0.md
Normal file
140
.changes/1.3.0.md
Normal file
@@ -0,0 +1,140 @@
|
||||
## dbt-core 1.3.0 - October 12, 2022
|
||||
|
||||
### Breaking Changes
|
||||
|
||||
- Renaming Metric Spec Attributes ([#5774](https://github.com/dbt-labs/dbt-core/issues/5774), [#5775](https://github.com/dbt-labs/dbt-core/pull/5775))
|
||||
|
||||
### Features
|
||||
|
||||
- Add `--defer` flag to dbt compile & dbt docs generate ([#4110](https://github.com/dbt-labs/dbt-core/issues/4110), [#4514](https://github.com/dbt-labs/dbt-core/pull/4514))
|
||||
- Python model inital version ([#5261](https://github.com/dbt-labs/dbt-core/issues/5261), [#5421](https://github.com/dbt-labs/dbt-core/pull/5421))
|
||||
- allows user to include the file extension for .py models in the dbt run -m command. ([#5289](https://github.com/dbt-labs/dbt-core/issues/5289), [#5295](https://github.com/dbt-labs/dbt-core/pull/5295))
|
||||
- Incremental materialization refactor and cleanup ([#5245](https://github.com/dbt-labs/dbt-core/issues/5245), [#5359](https://github.com/dbt-labs/dbt-core/pull/5359))
|
||||
- Python models can support incremental logic ([#0](https://github.com/dbt-labs/dbt-core/issues/0), [#35](https://github.com/dbt-labs/dbt-core/pull/35))
|
||||
- Add reusable function for retrying adapter connections. Utilize said function to add retries for Postgres (and Redshift). ([#5022](https://github.com/dbt-labs/dbt-core/issues/5022), [#5432](https://github.com/dbt-labs/dbt-core/pull/5432))
|
||||
- merge_exclude_columns for incremental materialization ([#5260](https://github.com/dbt-labs/dbt-core/issues/5260), [#5457](https://github.com/dbt-labs/dbt-core/pull/5457))
|
||||
- add exponential backoff to connection retries on Postgres (and Redshift) ([#5502](https://github.com/dbt-labs/dbt-core/issues/5502), [#5503](https://github.com/dbt-labs/dbt-core/pull/5503))
|
||||
- use MethodName.File when value ends with .csv ([#5578](https://github.com/dbt-labs/dbt-core/issues/5578), [#5581](https://github.com/dbt-labs/dbt-core/pull/5581))
|
||||
- Make `docs` configurable in `dbt_project.yml` and add a `node_color` attribute to change the color of nodes in the DAG ([#5333](https://github.com/dbt-labs/dbt-core/issues/5333), [#5397](https://github.com/dbt-labs/dbt-core/pull/5397))
|
||||
- Adding ResolvedMetricReference helper functions and tests ([#5567](https://github.com/dbt-labs/dbt-core/issues/5567), [#5607](https://github.com/dbt-labs/dbt-core/pull/5607))
|
||||
- Check dbt-core version requirements when installing Hub packages ([#5648](https://github.com/dbt-labs/dbt-core/issues/5648), [#5651](https://github.com/dbt-labs/dbt-core/pull/5651))
|
||||
- Search current working directory for `profiles.yml` ([#5411](https://github.com/dbt-labs/dbt-core/issues/5411), [#5717](https://github.com/dbt-labs/dbt-core/pull/5717))
|
||||
- Adding the `window` parameter to the metric spec. ([#5721](https://github.com/dbt-labs/dbt-core/issues/5721), [#5722](https://github.com/dbt-labs/dbt-core/pull/5722))
|
||||
- Add invocation args dict to ProviderContext class ([#5524](https://github.com/dbt-labs/dbt-core/issues/5524), [#5782](https://github.com/dbt-labs/dbt-core/pull/5782))
|
||||
- Adds new cli framework ([#5526](https://github.com/dbt-labs/dbt-core/issues/5526), [#5647](https://github.com/dbt-labs/dbt-core/pull/5647))
|
||||
- Flags work with new Click CLI ([#5529](https://github.com/dbt-labs/dbt-core/issues/5529), [#5790](https://github.com/dbt-labs/dbt-core/pull/5790))
|
||||
- Add metadata env method to ProviderContext class ([#5522](https://github.com/dbt-labs/dbt-core/issues/5522), [#5794](https://github.com/dbt-labs/dbt-core/pull/5794))
|
||||
- Array macros ([#5520](https://github.com/dbt-labs/dbt-core/issues/5520), [#5823](https://github.com/dbt-labs/dbt-core/pull/5823))
|
||||
- Add enabled config to exposures and metrics ([#5422](https://github.com/dbt-labs/dbt-core/issues/5422), [#5815](https://github.com/dbt-labs/dbt-core/pull/5815))
|
||||
- Migrate dbt-utils current_timestamp macros into core + adapters ([#5521](https://github.com/dbt-labs/dbt-core/issues/5521), [#5838](https://github.com/dbt-labs/dbt-core/pull/5838))
|
||||
- add -fr flag shorthand ([#5878](https://github.com/dbt-labs/dbt-core/issues/5878), [#5879](https://github.com/dbt-labs/dbt-core/pull/5879))
|
||||
- add type_boolean as a data type macro ([#5739](https://github.com/dbt-labs/dbt-core/issues/5739), [#5875](https://github.com/dbt-labs/dbt-core/pull/5875))
|
||||
- Support .dbtignore in project root to ignore certain files being read by dbt ([#5733](https://github.com/dbt-labs/dbt-core/issues/5733), [#5897](https://github.com/dbt-labs/dbt-core/pull/5897))
|
||||
- This conditionally no-ops warehouse connection at compile depending on an env var, disabling introspection/queries during compilation only. This is a temporary solution to more complex permissions requirements for the semantic layer. ([#5936](https://github.com/dbt-labs/dbt-core/issues/5936), [#5926](https://github.com/dbt-labs/dbt-core/pull/5926))
|
||||
|
||||
### Fixes
|
||||
|
||||
- Remove the default 256 characters limit on postgres character varying type when no limitation is set ([#5238](https://github.com/dbt-labs/dbt-core/issues/5238), [#5292](https://github.com/dbt-labs/dbt-core/pull/5292))
|
||||
- Include schema file config in unrendered_config ([#5338](https://github.com/dbt-labs/dbt-core/issues/5338), [#5344](https://github.com/dbt-labs/dbt-core/pull/5344))
|
||||
- Add context to compilation errors generated while rendering generic test configuration values. ([#5294](https://github.com/dbt-labs/dbt-core/issues/5294), [#5393](https://github.com/dbt-labs/dbt-core/pull/5393))
|
||||
- Resolves #5351 - Do not consider shorter varchar cols as schema changes ([#5351](https://github.com/dbt-labs/dbt-core/issues/5351), [#5395](https://github.com/dbt-labs/dbt-core/pull/5395))
|
||||
- Rename try to strict for more intuitiveness ([#5475](https://github.com/dbt-labs/dbt-core/issues/5475), [#5477](https://github.com/dbt-labs/dbt-core/pull/5477))
|
||||
- on_shchma_change fail verbosity enhancement ([#5504](https://github.com/dbt-labs/dbt-core/issues/5504), [#5505](https://github.com/dbt-labs/dbt-core/pull/5505))
|
||||
- Ignore empty strings passed in as secrets ([#5312](https://github.com/dbt-labs/dbt-core/issues/5312), [#5518](https://github.com/dbt-labs/dbt-core/pull/5518))
|
||||
- Fix handling of top-level exceptions ([#5564](https://github.com/dbt-labs/dbt-core/issues/5564), [#5560](https://github.com/dbt-labs/dbt-core/pull/5560))
|
||||
- Fix error rendering docs block in metrics description ([#5585](https://github.com/dbt-labs/dbt-core/issues/5585), [#5603](https://github.com/dbt-labs/dbt-core/pull/5603))
|
||||
- Extended validations for the project names ([#5379](https://github.com/dbt-labs/dbt-core/issues/5379), [#5620](https://github.com/dbt-labs/dbt-core/pull/5620))
|
||||
- Use sys.exit instead of exit ([#5621](https://github.com/dbt-labs/dbt-core/issues/5621), [#5627](https://github.com/dbt-labs/dbt-core/pull/5627))
|
||||
- Finishing logic upgrade to Redshift for name truncation collisions. ([#5586](https://github.com/dbt-labs/dbt-core/issues/5586), [#5656](https://github.com/dbt-labs/dbt-core/pull/5656))
|
||||
- multiple args for ref and source ([#5634](https://github.com/dbt-labs/dbt-core/issues/5634), [#5635](https://github.com/dbt-labs/dbt-core/pull/5635))
|
||||
- Fix Unexpected behavior when chaining methods on dbt-ref'ed/sourced dataframes ([#5646](https://github.com/dbt-labs/dbt-core/issues/5646), [#5677](https://github.com/dbt-labs/dbt-core/pull/5677))
|
||||
- Fix typos of comments in core/dbt/adapters/ ([#5690](https://github.com/dbt-labs/dbt-core/issues/5690), [#5693](https://github.com/dbt-labs/dbt-core/pull/5693))
|
||||
- Include py.typed in MANIFEST.in. This enables packages that install dbt-core from pypi to use mypy. ([#5703](https://github.com/dbt-labs/dbt-core/issues/5703), [#5703](https://github.com/dbt-labs/dbt-core/pull/5703))
|
||||
- Removal of all .coverage files when using make clean command ([#5633](https://github.com/dbt-labs/dbt-core/issues/5633), [#5759](https://github.com/dbt-labs/dbt-core/pull/5759))
|
||||
- Remove temp files generated by unit tests ([#5631](https://github.com/dbt-labs/dbt-core/issues/5631), [#5749](https://github.com/dbt-labs/dbt-core/pull/5749))
|
||||
- Fix warnings as errors during tests ([#5424](https://github.com/dbt-labs/dbt-core/issues/5424), [#5800](https://github.com/dbt-labs/dbt-core/pull/5800))
|
||||
- Prevent event_history from holding references ([#5848](https://github.com/dbt-labs/dbt-core/issues/5848), [#5858](https://github.com/dbt-labs/dbt-core/pull/5858))
|
||||
- Account for disabled flags on models in schema files more completely ([#3992](https://github.com/dbt-labs/dbt-core/issues/3992), [#5868](https://github.com/dbt-labs/dbt-core/pull/5868))
|
||||
- ConfigSelectorMethod should check for bools ([#5890](https://github.com/dbt-labs/dbt-core/issues/5890), [#5889](https://github.com/dbt-labs/dbt-core/pull/5889))
|
||||
- shorthand for full refresh should be one character ([#5878](https://github.com/dbt-labs/dbt-core/issues/5878), [#5908](https://github.com/dbt-labs/dbt-core/pull/5908))
|
||||
- Fix macro resolution order during static analysis for custom generic tests ([#5720](https://github.com/dbt-labs/dbt-core/issues/5720), [#5907](https://github.com/dbt-labs/dbt-core/pull/5907))
|
||||
- Fix race condition when invoking dbt via lib.py concurrently ([#5919](https://github.com/dbt-labs/dbt-core/issues/5919), [#5921](https://github.com/dbt-labs/dbt-core/pull/5921))
|
||||
- check length of args of python model function before accessing it ([#6041](https://github.com/dbt-labs/dbt-core/issues/6041), [#6042](https://github.com/dbt-labs/dbt-core/pull/6042))
|
||||
|
||||
### Docs
|
||||
|
||||
- Update dependency inline-source from ^6.1.5 to ^7.2.0 ([dbt-docs/#299](https://github.com/dbt-labs/dbt-docs/issues/299), [dbt-docs/#291](https://github.com/dbt-labs/dbt-docs/pull/291))
|
||||
- Update dependency jest from ^26.2.2 to ^28.1.3 ([dbt-docs/#299](https://github.com/dbt-labs/dbt-docs/issues/299), [dbt-docs/#291](https://github.com/dbt-labs/dbt-docs/pull/291))
|
||||
- Update dependency underscore from ^1.9.0 to ^1.13.4 ([dbt-docs/#299](https://github.com/dbt-labs/dbt-docs/issues/299), [dbt-docs/#291](https://github.com/dbt-labs/dbt-docs/pull/291))
|
||||
- Update dependency webpack-cli from ^3.3.12 to ^4.7.0 ([dbt-docs/#299](https://github.com/dbt-labs/dbt-docs/issues/299), [dbt-docs/#291](https://github.com/dbt-labs/dbt-docs/pull/291))
|
||||
- Update dependency webpack-dev-server from ^3.1.11 to ^4.9.3 ([dbt-docs/#299](https://github.com/dbt-labs/dbt-docs/issues/299), [dbt-docs/#291](https://github.com/dbt-labs/dbt-docs/pull/291))
|
||||
- Searches no longer require perfect matches, and instead consider each word individually. `my model` or `model my` will now find `my_model`, without the need for underscores ([dbt-docs/#143](https://github.com/dbt-labs/dbt-docs/issues/143), [dbt-docs/#145](https://github.com/dbt-labs/dbt-docs/pull/145))
|
||||
- Support the renaming of SQL to code happening in dbt-core ([dbt-docs/#299](https://github.com/dbt-labs/dbt-docs/issues/299), [dbt-docs/#292](https://github.com/dbt-labs/dbt-docs/pull/292))
|
||||
- Leverages `docs.node_color` from `dbt-core` to color nodes in the DAG ([dbt-docs/#44](https://github.com/dbt-labs/dbt-docs/issues/44), [dbt-docs/#281](https://github.com/dbt-labs/dbt-docs/pull/281))
|
||||
- Refer to exposures by their label by default. ([dbt-docs/#306](https://github.com/dbt-labs/dbt-docs/issues/306), [dbt-docs/#307](https://github.com/dbt-labs/dbt-docs/pull/307))
|
||||
|
||||
### Under the Hood
|
||||
|
||||
- Added language to tracked fields in run_model event ([#5571](https://github.com/dbt-labs/dbt-core/issues/5571), [#5469](https://github.com/dbt-labs/dbt-core/pull/5469))
|
||||
- Update mashumaro to 3.0.3 ([#4940](https://github.com/dbt-labs/dbt-core/issues/4940), [#5118](https://github.com/dbt-labs/dbt-core/pull/5118))
|
||||
- Add python incremental materialization test ([#0000](https://github.com/dbt-labs/dbt-core/issues/0000), [#5571](https://github.com/dbt-labs/dbt-core/pull/5571))
|
||||
- Save use of default env vars to manifest to enable partial parsing in those cases. ([#5155](https://github.com/dbt-labs/dbt-core/issues/5155), [#5589](https://github.com/dbt-labs/dbt-core/pull/5589))
|
||||
- add more information to log line interop test failures ([#5658](https://github.com/dbt-labs/dbt-core/issues/5658), [#5659](https://github.com/dbt-labs/dbt-core/pull/5659))
|
||||
- Add supported languages to materializations ([#5569](https://github.com/dbt-labs/dbt-core/issues/5569), [#5695](https://github.com/dbt-labs/dbt-core/pull/5695))
|
||||
- Migrate integration test 014 but also fix the snapshot hard delete test's timezone logic and force all integration tests to run flags.set_from_args to force environment variables are accessible to all integration test threads. ([#5760](https://github.com/dbt-labs/dbt-core/issues/5760), [#5760](https://github.com/dbt-labs/dbt-core/pull/5760))
|
||||
- Support dbt-metrics compilation by rebuilding flat_graph ([#5525](https://github.com/dbt-labs/dbt-core/issues/5525), [#5786](https://github.com/dbt-labs/dbt-core/pull/5786))
|
||||
- Reworking the way we define the window attribute of metrics to match freshness tests ([#5722](https://github.com/dbt-labs/dbt-core/issues/5722), [#5793](https://github.com/dbt-labs/dbt-core/pull/5793))
|
||||
- Add PythonJobHelper base class in core and add more type checking ([#5802](https://github.com/dbt-labs/dbt-core/issues/5802), [#5802](https://github.com/dbt-labs/dbt-core/pull/5802))
|
||||
- The link did not go to the anchor directly, now it does ([#5813](https://github.com/dbt-labs/dbt-core/issues/5813), [#5814](https://github.com/dbt-labs/dbt-core/pull/5814))
|
||||
- remove key as reserved keyword from test_bool_or ([#5817](https://github.com/dbt-labs/dbt-core/issues/5817), [#5818](https://github.com/dbt-labs/dbt-core/pull/5818))
|
||||
- Convert default selector tests to pytest ([#5728](https://github.com/dbt-labs/dbt-core/issues/5728), [#5820](https://github.com/dbt-labs/dbt-core/pull/5820))
|
||||
- Compatibiltiy for metric attribute renaming ([#5807](https://github.com/dbt-labs/dbt-core/issues/5807), [#5825](https://github.com/dbt-labs/dbt-core/pull/5825))
|
||||
- remove source quoting setting in adapter tests ([#5836](https://github.com/dbt-labs/dbt-core/issues/5836), [#5839](https://github.com/dbt-labs/dbt-core/pull/5839))
|
||||
- Add name validation for metrics ([#5456](https://github.com/dbt-labs/dbt-core/issues/5456), [#5841](https://github.com/dbt-labs/dbt-core/pull/5841))
|
||||
- Validate exposure name and add label ([#5606](https://github.com/dbt-labs/dbt-core/issues/5606), [#5844](https://github.com/dbt-labs/dbt-core/pull/5844))
|
||||
- Adding validation for metric expression attribute ([#5871](https://github.com/dbt-labs/dbt-core/issues/5871), [#5873](https://github.com/dbt-labs/dbt-core/pull/5873))
|
||||
- Profiling and Adapter Management work with Click CLI ([#5531](https://github.com/dbt-labs/dbt-core/issues/5531), [#5892](https://github.com/dbt-labs/dbt-core/pull/5892))
|
||||
- Reparse references to deleted metric ([#5444](https://github.com/dbt-labs/dbt-core/issues/5444), [#5920](https://github.com/dbt-labs/dbt-core/pull/5920))
|
||||
|
||||
### Dependencies
|
||||
|
||||
- Upgrade to Jinja2==3.1.2 from Jinja2==2.11.3 ([#4748](https://github.com/dbt-labs/dbt-core/issues/4748), [#5465](https://github.com/dbt-labs/dbt-core/pull/5465))
|
||||
- Bump mypy from 0.961 to 0.971 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5495](https://github.com/dbt-labs/dbt-core/pull/5495))
|
||||
- Remove pin for MarkUpSafe from >=0.23,<2.1 ([#5506](https://github.com/dbt-labs/dbt-core/issues/5506), [#5507](https://github.com/dbt-labs/dbt-core/pull/5507))
|
||||
|
||||
### Dependency
|
||||
|
||||
- Bump python from 3.10.5-slim-bullseye to 3.10.6-slim-bullseye in /docker ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5623](https://github.com/dbt-labs/dbt-core/pull/5623))
|
||||
- Bump mashumaro[msgpack] from 3.0.3 to 3.0.4 in /core ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5649](https://github.com/dbt-labs/dbt-core/pull/5649))
|
||||
- Bump black from 22.6.0 to 22.8.0 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5750](https://github.com/dbt-labs/dbt-core/pull/5750))
|
||||
- Bump python from 3.10.6-slim-bullseye to 3.10.7-slim-bullseye in /docker ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5805](https://github.com/dbt-labs/dbt-core/pull/5805))
|
||||
|
||||
### Contributors
|
||||
- [@Goodkat](https://github.com/Goodkat) ([#5581](https://github.com/dbt-labs/dbt-core/pull/5581), [#5518](https://github.com/dbt-labs/dbt-core/pull/5518), [#5620](https://github.com/dbt-labs/dbt-core/pull/5620))
|
||||
- [@Ilanbenb](https://github.com/Ilanbenb) ([#5505](https://github.com/dbt-labs/dbt-core/pull/5505))
|
||||
- [@b-per](https://github.com/b-per) ([#5397](https://github.com/dbt-labs/dbt-core/pull/5397), [dbt-docs/#281](https://github.com/dbt-labs/dbt-docs/pull/281))
|
||||
- [@bbroeksema](https://github.com/bbroeksema) ([#5749](https://github.com/dbt-labs/dbt-core/pull/5749))
|
||||
- [@callum-mcdata](https://github.com/callum-mcdata) ([#5775](https://github.com/dbt-labs/dbt-core/pull/5775), [#5607](https://github.com/dbt-labs/dbt-core/pull/5607), [#5722](https://github.com/dbt-labs/dbt-core/pull/5722), [#5793](https://github.com/dbt-labs/dbt-core/pull/5793), [#5825](https://github.com/dbt-labs/dbt-core/pull/5825), [#5873](https://github.com/dbt-labs/dbt-core/pull/5873))
|
||||
- [@danielcmessias](https://github.com/danielcmessias) ([#5889](https://github.com/dbt-labs/dbt-core/pull/5889))
|
||||
- [@dave-connors-3](https://github.com/dave-connors-3) ([#5457](https://github.com/dbt-labs/dbt-core/pull/5457), [#5879](https://github.com/dbt-labs/dbt-core/pull/5879), [#5908](https://github.com/dbt-labs/dbt-core/pull/5908))
|
||||
- [@dbeatty10](https://github.com/dbeatty10) ([#5717](https://github.com/dbt-labs/dbt-core/pull/5717), [#5823](https://github.com/dbt-labs/dbt-core/pull/5823))
|
||||
- [@drewbanin](https://github.com/drewbanin) ([#5921](https://github.com/dbt-labs/dbt-core/pull/5921), [dbt-docs/#292](https://github.com/dbt-labs/dbt-docs/pull/292))
|
||||
- [@epapineau](https://github.com/epapineau) ([#5395](https://github.com/dbt-labs/dbt-core/pull/5395))
|
||||
- [@graciegoheen](https://github.com/graciegoheen) ([#5823](https://github.com/dbt-labs/dbt-core/pull/5823))
|
||||
- [@jared-rimmer](https://github.com/jared-rimmer) ([#5782](https://github.com/dbt-labs/dbt-core/pull/5782), [#5794](https://github.com/dbt-labs/dbt-core/pull/5794), [#5759](https://github.com/dbt-labs/dbt-core/pull/5759))
|
||||
- [@jeremyyeo](https://github.com/jeremyyeo) ([#5477](https://github.com/dbt-labs/dbt-core/pull/5477))
|
||||
- [@joellabes](https://github.com/joellabes) ([dbt-docs/#145](https://github.com/dbt-labs/dbt-docs/pull/145))
|
||||
- [@jpmmcneill](https://github.com/jpmmcneill) ([#5875](https://github.com/dbt-labs/dbt-core/pull/5875))
|
||||
- [@kadero](https://github.com/kadero) ([#4514](https://github.com/dbt-labs/dbt-core/pull/4514))
|
||||
- [@leoebfolsom](https://github.com/leoebfolsom) ([#5295](https://github.com/dbt-labs/dbt-core/pull/5295))
|
||||
- [@matt-winkler](https://github.com/matt-winkler) ([#5397](https://github.com/dbt-labs/dbt-core/pull/5397), [dbt-docs/#281](https://github.com/dbt-labs/dbt-docs/pull/281))
|
||||
- [@nicholasyager](https://github.com/nicholasyager) ([#5393](https://github.com/dbt-labs/dbt-core/pull/5393))
|
||||
- [@panasenco](https://github.com/panasenco) ([#5703](https://github.com/dbt-labs/dbt-core/pull/5703))
|
||||
- [@racheldaniel](https://github.com/racheldaniel) ([#5926](https://github.com/dbt-labs/dbt-core/pull/5926))
|
||||
- [@sdebruyn](https://github.com/sdebruyn) ([#5814](https://github.com/dbt-labs/dbt-core/pull/5814), [#5818](https://github.com/dbt-labs/dbt-core/pull/5818), [#5839](https://github.com/dbt-labs/dbt-core/pull/5839))
|
||||
- [@shrodingers](https://github.com/shrodingers) ([#5292](https://github.com/dbt-labs/dbt-core/pull/5292))
|
||||
- [@sungchun12](https://github.com/sungchun12) ([#5397](https://github.com/dbt-labs/dbt-core/pull/5397), [dbt-docs/#281](https://github.com/dbt-labs/dbt-docs/pull/281))
|
||||
- [@tomasfarias](https://github.com/tomasfarias) ([#5432](https://github.com/dbt-labs/dbt-core/pull/5432))
|
||||
- [@varun-dc](https://github.com/varun-dc) ([#5627](https://github.com/dbt-labs/dbt-core/pull/5627))
|
||||
- [@yoiki](https://github.com/yoiki) ([#5693](https://github.com/dbt-labs/dbt-core/pull/5693))
|
||||
- [@chamini2](https://github.com/chamini2) ([#6042](https://github.com/dbt-labs/dbt-core/pull/6042))
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Dependencies
|
||||
body: Upgrade to Jinja2==3.1.2 from Jinja2==2.11.3
|
||||
time: 2022-07-19T14:16:46.665631-05:00
|
||||
custom:
|
||||
Author: emmyoop
|
||||
Issue: "4748"
|
||||
PR: "5465"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Dependencies
|
||||
body: "Bump mypy from 0.961 to 0.971"
|
||||
time: 2022-07-20T00:07:41.000000-05:00
|
||||
custom:
|
||||
Author: dependabot[bot]
|
||||
Issue: "4904"
|
||||
PR: "5495"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Dependencies
|
||||
body: Remove pin for MarkUpSafe from >=0.23,<2.1
|
||||
time: 2022-07-21T09:32:33.494002-05:00
|
||||
custom:
|
||||
Author: emmyoop
|
||||
Issue: "5506"
|
||||
PR: "5507"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Dependency
|
||||
body: "Bump python from 3.10.5-slim-bullseye to 3.10.6-slim-bullseye in /docker"
|
||||
time: 2022-08-08T13:23:27.00000Z
|
||||
custom:
|
||||
Author: dependabot[bot]
|
||||
Issue: 4904
|
||||
PR: 5623
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Dependency
|
||||
body: "Bump mashumaro[msgpack] from 3.0.3 to 3.0.4 in /core"
|
||||
time: 2022-08-15T18:27:31.00000Z
|
||||
custom:
|
||||
Author: dependabot[bot]
|
||||
Issue: 4904
|
||||
PR: 5649
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Docs
|
||||
body: Update dependency inline-source from ^6.1.5 to ^7.2.0
|
||||
time: 2022-07-28T14:02:58.441963-07:00
|
||||
custom:
|
||||
Author: emmyoop
|
||||
Issue: "299"
|
||||
PR: "291"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Docs
|
||||
body: Update dependency jest from ^26.2.2 to ^28.1.3
|
||||
time: 2022-07-28T14:03:29.837274-07:00
|
||||
custom:
|
||||
Author: emmyoop
|
||||
Issue: "299"
|
||||
PR: "291"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Docs
|
||||
body: Update dependency underscore from ^1.9.0 to ^1.13.4
|
||||
time: 2022-07-28T14:03:51.123441-07:00
|
||||
custom:
|
||||
Author: emmyoop
|
||||
Issue: "299"
|
||||
PR: "291"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Docs
|
||||
body: Update dependency webpack-cli from ^3.3.12 to ^4.7.0
|
||||
time: 2022-07-28T14:04:25.629638-07:00
|
||||
custom:
|
||||
Author: emmyoop
|
||||
Issue: "299"
|
||||
PR: "291"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Docs
|
||||
body: Update dependency webpack-dev-server from ^3.1.11 to ^4.9.3
|
||||
time: 2022-07-28T14:04:49.637369-07:00
|
||||
custom:
|
||||
Author: emmyoop
|
||||
Issue: "299"
|
||||
PR: "291"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Docs
|
||||
body: Searches no longer require perfect matches, and instead consider each word individually.
|
||||
`my model` or `model my` will now find `my_model`, without the need for underscores
|
||||
time: 2022-07-28T14:06:20.371364-07:00
|
||||
custom:
|
||||
Author: joellabes
|
||||
Issue: "143"
|
||||
PR: "145"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Docs
|
||||
body: Support the renaming of SQL to code happening in dbt-core
|
||||
time: 2022-07-28T14:08:06.184934-07:00
|
||||
custom:
|
||||
Author: jtcohen6 stu-k drewbanin ChenyuLInx
|
||||
Issue: "299"
|
||||
PR: "292"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Docs
|
||||
body: Leverages `docs.node_color` from `dbt-core` to color nodes in the DAG
|
||||
time: 2022-08-04T13:41:38.669987-05:00
|
||||
custom:
|
||||
Author: matt-winkler sungchun12 b-per
|
||||
Issue: "44"
|
||||
PR: "281"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Add `--defer` flag to dbt compile & dbt docs generate
|
||||
time: 2022-03-14T18:11:27.783107864+01:00
|
||||
custom:
|
||||
Author: kadero
|
||||
Issue: "4110"
|
||||
PR: "4514"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Python model inital version
|
||||
time: 2022-05-10T16:51:30.245589-07:00
|
||||
custom:
|
||||
Author: ChenyuLInx
|
||||
Issue: "5261"
|
||||
PR: "5421"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: allows user to include the file extension for .py models in the dbt run -m command.
|
||||
time: 2022-05-26T16:53:23.389671-07:00
|
||||
custom:
|
||||
Author: leoebfolsom
|
||||
Issue: "5289"
|
||||
PR: "5295"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Incremental materialization refactor and cleanup
|
||||
time: 2022-06-10T10:56:47.226887-04:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "5245"
|
||||
PR: "5359"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Python models can support incremental logic
|
||||
time: 2022-06-27T13:10:42.123303-05:00
|
||||
custom:
|
||||
Author: iknox-fa
|
||||
Issue: "0"
|
||||
PR: "35"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Features
|
||||
body: Add reusable function for retrying adapter connections. Utilize said function
|
||||
to add retries for Postgres (and Redshift).
|
||||
time: 2022-07-15T03:55:55.270637265+02:00
|
||||
custom:
|
||||
Author: tomasfarias
|
||||
Issue: "5022"
|
||||
PR: "5432"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: add exponential backoff to connection retries on Postgres (and Redshift)
|
||||
time: 2022-07-20T17:12:57.486949-04:00
|
||||
custom:
|
||||
Author: nathaniel-may
|
||||
Issue: "5502"
|
||||
PR: "5503"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: use MethodName.File when value ends with .csv
|
||||
time: 2022-07-29T17:32:31.395677157+02:00
|
||||
custom:
|
||||
Author: Goodkat
|
||||
Issue: "5578"
|
||||
PR: "5581"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Features
|
||||
body: Make `docs` configurable in `dbt_project.yml` and add a `node_color` attribute
|
||||
to change the color of nodes in the DAG
|
||||
time: 2022-08-03T10:42:30.60624+02:00
|
||||
custom:
|
||||
Author: matt-winkler sungchun12 b-per
|
||||
Issue: "5333"
|
||||
PR: "5397"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Adding ResolvedMetricReference helper functions and tests
|
||||
time: 2022-08-04T12:09:36.202919-04:00
|
||||
custom:
|
||||
Author: callum-mcdata
|
||||
Issue: "5567"
|
||||
PR: "5607"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Check dbt-core version requirements when installing Hub packages
|
||||
time: 2022-08-15T13:43:12.965143+01:00
|
||||
custom:
|
||||
Author: jtcohen6
|
||||
Issue: "5648"
|
||||
PR: "5651"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Remove the default 256 characters limit on postgres character varying type when
|
||||
no limitation is set
|
||||
time: 2022-05-23T10:38:43.392232+02:00
|
||||
custom:
|
||||
Author: shrodingers
|
||||
Issue: "5238"
|
||||
PR: "5292"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Include schema file config in unrendered_config
|
||||
time: 2022-06-07T12:30:58.535207-04:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "5338"
|
||||
PR: "5344"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: 'Resolves #5351 - Do not consider shorter varchar cols as schema changes'
|
||||
time: 2022-06-17T19:37:31.885484-07:00
|
||||
custom:
|
||||
Author: epapineau
|
||||
Issue: "5351"
|
||||
PR: "5395"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Rename try to strict for more intuitiveness
|
||||
time: 2022-07-15T23:11:48.327928+12:00
|
||||
custom:
|
||||
Author: jeremyyeo
|
||||
Issue: "5475"
|
||||
PR: "5477"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: on_shchma_change fail verbosity enhancement
|
||||
time: 2022-07-20T21:37:46.474241+03:00
|
||||
custom:
|
||||
Author: Ilanbenb
|
||||
Issue: "5504"
|
||||
PR: "5505"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Ignore empty strings passed in as secrets
|
||||
time: 2022-07-23T21:53:30.907759094+02:00
|
||||
custom:
|
||||
Author: Goodkat
|
||||
Issue: "5312"
|
||||
PR: "5518"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Fix handling of top-level exceptions
|
||||
time: 2022-07-26T11:36:36.824979-04:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "5564"
|
||||
PR: "5560"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Fix error rendering docs block in metrics description
|
||||
time: 2022-08-03T14:42:21.386265-04:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "5585"
|
||||
PR: "5603"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Extended validations for the project names
|
||||
time: 2022-08-05T22:10:22.746830854+02:00
|
||||
custom:
|
||||
Author: Goodkat
|
||||
Issue: "5379"
|
||||
PR: "5620"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Use sys.exit instead of exit
|
||||
time: 2022-08-08T11:20:01.838171926-04:00
|
||||
custom:
|
||||
Author: varun-dc
|
||||
Issue: "5621"
|
||||
PR: "5627"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Finishing logic upgrade to Redshift for name truncation collisions.
|
||||
time: 2022-08-15T23:04:09.173645-07:00
|
||||
custom:
|
||||
Author: versusfacit
|
||||
Issue: "5586"
|
||||
PR: "5656"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: multiple args for ref and source
|
||||
time: 2022-08-16T15:34:01.348339-07:00
|
||||
custom:
|
||||
Author: ChenyuLInx
|
||||
Issue: "5634"
|
||||
PR: "5635"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Fix Unexpected behavior when chaining methods on dbt-ref'ed/sourced dataframes
|
||||
time: 2022-08-17T16:36:42.678275-07:00
|
||||
custom:
|
||||
Author: ChenyuLInx
|
||||
Issue: "5646"
|
||||
PR: "5677"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Add context to compilation errors generated while rendering generic test configuration
|
||||
values.
|
||||
time: 2022-06-17T15:07:44.751037-04:00
|
||||
custom:
|
||||
Author: nicholasyager
|
||||
Issue: "5294"
|
||||
PR: "5393"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Added language to tracked fields in run_model event
|
||||
time: 2022-07-13T12:49:25.362678-05:00
|
||||
custom:
|
||||
Author: stu-k
|
||||
Issue: "5571"
|
||||
PR: "5469"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Update mashumaro to 3.0.3
|
||||
time: 2022-07-20T11:52:26.210876-04:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "4940"
|
||||
PR: "5118"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Add python incremental materialization test
|
||||
time: 2022-07-28T09:45:36.13608-05:00
|
||||
custom:
|
||||
Author: stu-k
|
||||
Issue: "0000"
|
||||
PR: "5571"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Save use of default env vars to manifest to enable partial parsing in those
|
||||
cases.
|
||||
time: 2022-08-02T11:29:36.417589-05:00
|
||||
custom:
|
||||
Author: emmyoop
|
||||
Issue: "5155"
|
||||
PR: "5589"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: add more information to log line interop test failures
|
||||
time: 2022-08-16T12:20:32.119588+01:00
|
||||
custom:
|
||||
Author: nathaniel-may
|
||||
Issue: "5658"
|
||||
PR: "5659"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Add supported languages to materializations
|
||||
time: 2022-08-22T10:37:39.50743-05:00
|
||||
custom:
|
||||
Author: stu-k
|
||||
Issue: "5569"
|
||||
PR: "5695"
|
||||
10
.changes/1.3.1.md
Normal file
10
.changes/1.3.1.md
Normal file
@@ -0,0 +1,10 @@
|
||||
## dbt-core 1.3.1 - November 16, 2022
|
||||
### Features
|
||||
- This pulls the profile name from args when constructing a RuntimeConfig in lib.py, enabling the dbt-server to override the value that's in the dbt_project.yml ([#6201](https://github.com/dbt-labs/dbt-core/issues/6201), [#6202](https://github.com/dbt-labs/dbt-core/pull/6202))
|
||||
### Docs
|
||||
- ([dbt-docs/#5880](https://github.com/dbt-labs/dbt-docs/issues/5880), [dbt-docs/#324](https://github.com/dbt-labs/dbt-docs/pull/324))
|
||||
- Fix rendering of sample code for metrics ([dbt-docs/#323](https://github.com/dbt-labs/dbt-docs/issues/323), [dbt-docs/#346](https://github.com/dbt-labs/dbt-docs/pull/346))
|
||||
|
||||
### Contributors
|
||||
- [@paulbenschmidt](https://github.com/paulbenschmidt) ([dbt-docs/#324](https://github.com/dbt-labs/dbt-docs/pull/324))
|
||||
- [@racheldaniel](https://github.com/racheldaniel) ([#6202](https://github.com/dbt-labs/dbt-core/pull/6202))
|
||||
5
.changes/1.3.2.md
Normal file
5
.changes/1.3.2.md
Normal file
@@ -0,0 +1,5 @@
|
||||
## dbt-core 1.3.2 - January 04, 2023
|
||||
|
||||
### Fixes
|
||||
|
||||
- Bug when partial parsing with an empty schema file ([#4850](https://github.com/dbt-labs/dbt-core/issues/4850), [#<no value>](https://github.com/dbt-labs/dbt-core/pull/<no value>))
|
||||
8
.changes/1.3.3.md
Normal file
8
.changes/1.3.3.md
Normal file
@@ -0,0 +1,8 @@
|
||||
## dbt-core 1.3.3 - February 28, 2023
|
||||
|
||||
### Fixes
|
||||
|
||||
- add pytz dependency ([#7077](https://github.com/dbt-labs/dbt-core/issues/7077))
|
||||
|
||||
### Contributors
|
||||
- [@sdebruyn](https://github.com/sdebruyn) ([#7077](https://github.com/dbt-labs/dbt-core/issues/7077))
|
||||
8
.changes/1.3.4.md
Normal file
8
.changes/1.3.4.md
Normal file
@@ -0,0 +1,8 @@
|
||||
## dbt-core 1.3.4 - April 19, 2023
|
||||
|
||||
### Fixes
|
||||
|
||||
- Improved failed event serialization handling and associated tests ([#7113](https://github.com/dbt-labs/dbt-core/issues/7113), [#7108](https://github.com/dbt-labs/dbt-core/issues/7108), [#6568](https://github.com/dbt-labs/dbt-core/issues/6568))
|
||||
|
||||
### Contributors
|
||||
- [@QMalcolm](https://github.com/QMalcolm) ([#7113](https://github.com/dbt-labs/dbt-core/issues/7113), [#7108](https://github.com/dbt-labs/dbt-core/issues/7108), [#6568](https://github.com/dbt-labs/dbt-core/issues/6568))
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Breaking Changes
|
||||
body: Renaming Metric Spec Attributes
|
||||
time: 2022-09-06T15:45:21.2769-05:00
|
||||
custom:
|
||||
Author: callum-mcdata
|
||||
Issue: "5774"
|
||||
PR: "5775"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: "Dependency"
|
||||
body: "Bump black from 22.6.0 to 22.8.0"
|
||||
time: 2022-09-06T13:48:58.00000Z
|
||||
custom:
|
||||
Author: dependabot[bot]
|
||||
Issue: 4904
|
||||
PR: 5750
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: "Dependency"
|
||||
body: "Bump python from 3.10.6-slim-bullseye to 3.10.7-slim-bullseye in /docker"
|
||||
time: 2022-09-12T00:22:53.00000Z
|
||||
custom:
|
||||
Author: dependabot[bot]
|
||||
Issue: 4904
|
||||
PR: 5805
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Adding the `window` parameter to the metric spec.
|
||||
time: 2022-08-31T12:13:19.48356-05:00
|
||||
custom:
|
||||
Author: callum-mcdata
|
||||
Issue: "5721"
|
||||
PR: "5722"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Add invocation args dict to ProviderContext class
|
||||
time: 2022-09-08T08:13:15.17337+01:00
|
||||
custom:
|
||||
Author: jared-rimmer
|
||||
Issue: "5524"
|
||||
PR: "5782"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Adds new cli framework
|
||||
time: 2022-09-08T10:41:49.375734-05:00
|
||||
custom:
|
||||
Author: iknox-fa
|
||||
Issue: "5526"
|
||||
PR: "5647"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Features
|
||||
body: Add enabled config to exposures and metrics
|
||||
time: 2022-09-13T09:59:24.445918-05:00
|
||||
custom:
|
||||
Author: emmyoop
|
||||
Issue: "5422"
|
||||
PR: "5815"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Include py.typed in MANIFEST.in. This enables packages that install dbt-core
|
||||
from pypi to use mypy.
|
||||
time: 2022-08-23T11:26:33.8415455-07:00
|
||||
custom:
|
||||
Author: panasenco
|
||||
Issue: "5703"
|
||||
PR: "5703"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Removal of all .coverage files when using make clean command
|
||||
time: 2022-09-03T15:54:09.741554082+01:00
|
||||
custom:
|
||||
Author: jared-rimmer
|
||||
Issue: "5633"
|
||||
PR: "5759"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Remove temp files generated by unit tests
|
||||
time: 2022-09-09T09:28:32.590208+02:00
|
||||
custom:
|
||||
Author: bbroeksema
|
||||
Issue: "5631"
|
||||
PR: "5749"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Fixes
|
||||
body: Fix warnings as errors during tests
|
||||
time: 2022-09-09T09:56:27.90654-05:00
|
||||
custom:
|
||||
Author: stu-k
|
||||
Issue: "5424"
|
||||
PR: "5800"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Migrate integration test 014 but also fix the snapshot hard delete test's timezone logic and force all integration tests to run flags.set_from_args to force environment variables are accessible to all integration test threads.
|
||||
time: 2022-09-05T00:17:49.564534-07:00
|
||||
custom:
|
||||
Author: versusfacit
|
||||
Issue: "5760"
|
||||
PR: "5760"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Support dbt-metrics compilation by rebuilding flat_graph
|
||||
time: 2022-09-08T14:56:44.173322-04:00
|
||||
custom:
|
||||
Author: gshank
|
||||
Issue: "5525"
|
||||
PR: "5786"
|
||||
@@ -1,8 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Reworking the way we define the window attribute of metrics to match freshness
|
||||
tests
|
||||
time: 2022-09-08T18:07:31.532608-05:00
|
||||
custom:
|
||||
Author: callum-mcdata
|
||||
Issue: "5722"
|
||||
PR: "5793"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Add PythonJobHelper base class in core and add more type checking
|
||||
time: 2022-09-09T11:52:20.419364-07:00
|
||||
custom:
|
||||
Author: ChenyuLInx
|
||||
Issue: "5802"
|
||||
PR: "5802"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: Convert default selector tests to pytest
|
||||
time: 2022-09-12T13:40:00.625912-05:00
|
||||
custom:
|
||||
Author: stu-k
|
||||
Issue: "5728"
|
||||
PR: "5820"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: The link did not go to the anchor directly, now it does
|
||||
time: 2022-09-12T14:00:35.899828+02:00
|
||||
custom:
|
||||
Author: sdebruyn
|
||||
Issue: "5813"
|
||||
PR: "5814"
|
||||
@@ -1,7 +0,0 @@
|
||||
kind: Under the Hood
|
||||
body: remove key as reserved keyword from test_bool_or
|
||||
time: 2022-09-12T19:03:41.481601+02:00
|
||||
custom:
|
||||
Author: sdebruyn
|
||||
Issue: "5817"
|
||||
PR: "5818"
|
||||
113
.changie.yaml
Executable file → Normal file
113
.changie.yaml
Executable file → Normal file
@@ -6,19 +6,67 @@ changelogPath: CHANGELOG.md
|
||||
versionExt: md
|
||||
versionFormat: '## dbt-core {{.Version}} - {{.Time.Format "January 02, 2006"}}'
|
||||
kindFormat: '### {{.Kind}}'
|
||||
changeFormat: '- {{.Body}} ([#{{.Custom.Issue}}](https://github.com/dbt-labs/dbt-core/issues/{{.Custom.Issue}}), [#{{.Custom.PR}}](https://github.com/dbt-labs/dbt-core/pull/{{.Custom.PR}}))'
|
||||
changeFormat: |-
|
||||
{{- $IssueList := list }}
|
||||
{{- $changes := splitList " " $.Custom.Issue }}
|
||||
{{- range $issueNbr := $changes }}
|
||||
{{- $changeLink := "[#nbr](https://github.com/dbt-labs/dbt-core/issues/nbr)" | replace "nbr" $issueNbr }}
|
||||
{{- $IssueList = append $IssueList $changeLink }}
|
||||
{{- end -}}
|
||||
- {{.Body}} ({{ range $index, $element := $IssueList }}{{if $index}}, {{end}}{{$element}}{{end}})
|
||||
|
||||
kinds:
|
||||
- label: Breaking Changes
|
||||
- label: Features
|
||||
- label: Fixes
|
||||
- label: Docs
|
||||
changeFormat: '- {{.Body}} ([dbt-docs/#{{.Custom.Issue}}](https://github.com/dbt-labs/dbt-docs/issues/{{.Custom.Issue}}), [dbt-docs/#{{.Custom.PR}}](https://github.com/dbt-labs/dbt-docs/pull/{{.Custom.PR}}))'
|
||||
changeFormat: |-
|
||||
{{- $IssueList := list }}
|
||||
{{- $changes := splitList " " $.Custom.Issue }}
|
||||
{{- range $issueNbr := $changes }}
|
||||
{{- $changeLink := "[dbt-docs/#nbr](https://github.com/dbt-labs/dbt-docs/issues/nbr)" | replace "nbr" $issueNbr }}
|
||||
{{- $IssueList = append $IssueList $changeLink }}
|
||||
{{- end -}}
|
||||
- {{.Body}} ({{ range $index, $element := $IssueList }}{{if $index}}, {{end}}{{$element}}{{end}})
|
||||
- label: Under the Hood
|
||||
- label: Dependencies
|
||||
changeFormat: '- {{.Body}} ({{if ne .Custom.Issue ""}}[#{{.Custom.Issue}}](https://github.com/dbt-labs/dbt-core/issues/{{.Custom.Issue}}), {{end}}[#{{.Custom.PR}}](https://github.com/dbt-labs/dbt-core/pull/{{.Custom.PR}}))'
|
||||
changeFormat: |-
|
||||
{{- $PRList := list }}
|
||||
{{- $changes := splitList " " $.Custom.PR }}
|
||||
{{- range $pullrequest := $changes }}
|
||||
{{- $changeLink := "[#nbr](https://github.com/dbt-labs/dbt-core/pull/nbr)" | replace "nbr" $pullrequest }}
|
||||
{{- $PRList = append $PRList $changeLink }}
|
||||
{{- end -}}
|
||||
- {{.Body}} ({{ range $index, $element := $PRList }}{{if $index}}, {{end}}{{$element}}{{end}})
|
||||
skipGlobalChoices: true
|
||||
additionalChoices:
|
||||
- key: Author
|
||||
label: GitHub Username(s) (separated by a single space if multiple)
|
||||
type: string
|
||||
minLength: 3
|
||||
- key: PR
|
||||
label: GitHub Pull Request Number (separated by a single space if multiple)
|
||||
type: string
|
||||
minLength: 1
|
||||
- label: Security
|
||||
changeFormat: '- {{.Body}} ({{if ne .Custom.Issue ""}}[#{{.Custom.Issue}}](https://github.com/dbt-labs/dbt-core/issues/{{.Custom.Issue}}), {{end}}[#{{.Custom.PR}}](https://github.com/dbt-labs/dbt-core/pull/{{.Custom.PR}}))'
|
||||
changeFormat: |-
|
||||
{{- $PRList := list }}
|
||||
{{- $changes := splitList " " $.Custom.PR }}
|
||||
{{- range $pullrequest := $changes }}
|
||||
{{- $changeLink := "[#nbr](https://github.com/dbt-labs/dbt-core/pull/nbr)" | replace "nbr" $pullrequest }}
|
||||
{{- $PRList = append $PRList $changeLink }}
|
||||
{{- end -}}
|
||||
- {{.Body}} ({{ range $index, $element := $PRList }}{{if $index}}, {{end}}{{$element}}{{end}})
|
||||
skipGlobalChoices: true
|
||||
additionalChoices:
|
||||
- key: Author
|
||||
label: GitHub Username(s) (separated by a single space if multiple)
|
||||
type: string
|
||||
minLength: 3
|
||||
- key: PR
|
||||
label: GitHub Pull Request Number (separated by a single space if multiple)
|
||||
type: string
|
||||
minLength: 1
|
||||
|
||||
newlines:
|
||||
afterChangelogHeader: 1
|
||||
@@ -33,42 +81,47 @@ custom:
|
||||
type: string
|
||||
minLength: 3
|
||||
- key: Issue
|
||||
label: GitHub Issue Number
|
||||
type: int
|
||||
minInt: 1
|
||||
- key: PR
|
||||
label: GitHub Pull Request Number
|
||||
type: int
|
||||
minInt: 1
|
||||
label: GitHub Issue Number (separated by a single space if multiple)
|
||||
type: string
|
||||
minLength: 1
|
||||
|
||||
footerFormat: |
|
||||
{{- $contributorDict := dict }}
|
||||
{{- /* any names added to this list should be all lowercase for later matching purposes */}}
|
||||
{{- $core_team := list "emmyoop" "nathaniel-may" "gshank" "leahwicz" "chenyulinx" "stu-k" "iknox-fa" "versusfacit" "mcknight-42" "jtcohen6" "dependabot[bot]" "snyk-bot" }}
|
||||
{{- $core_team := list "michelleark" "peterallenwebb" "emmyoop" "nathaniel-may" "gshank" "leahwicz" "chenyulinx" "stu-k" "iknox-fa" "versusfacit" "mcknight-42" "jtcohen6" "aranke" "dependabot[bot]" "snyk-bot" "colin-rogers-dbt" }}
|
||||
{{- range $change := .Changes }}
|
||||
{{- $authorList := splitList " " $change.Custom.Author }}
|
||||
{{- /* loop through all authors for a PR */}}
|
||||
{{- /* loop through all authors for a single changelog */}}
|
||||
{{- range $author := $authorList }}
|
||||
{{- $authorLower := lower $author }}
|
||||
{{- /* we only want to include non-core team contributors */}}
|
||||
{{- if not (has $authorLower $core_team)}}
|
||||
{{- /* Docs kind link back to dbt-docs instead of dbt-core PRs */}}
|
||||
{{- $prLink := $change.Kind }}
|
||||
{{- if eq $change.Kind "Docs" }}
|
||||
{{- $prLink = "[dbt-docs/#pr](https://github.com/dbt-labs/dbt-docs/pull/pr)" | replace "pr" $change.Custom.PR }}
|
||||
{{- else }}
|
||||
{{- $prLink = "[#pr](https://github.com/dbt-labs/dbt-core/pull/pr)" | replace "pr" $change.Custom.PR }}
|
||||
{{- end }}
|
||||
{{- /* check if this contributor has other PRs associated with them already */}}
|
||||
{{- if hasKey $contributorDict $author }}
|
||||
{{- $prList := get $contributorDict $author }}
|
||||
{{- $prList = append $prList $prLink }}
|
||||
{{- $contributorDict := set $contributorDict $author $prList }}
|
||||
{{- else }}
|
||||
{{- $prList := list $prLink }}
|
||||
{{- $contributorDict := set $contributorDict $author $prList }}
|
||||
{{- end }}
|
||||
{{- end}}
|
||||
{{- $changeList := splitList " " $change.Custom.Author }}
|
||||
{{- $IssueList := list }}
|
||||
{{- $changeLink := $change.Kind }}
|
||||
{{- if or (eq $change.Kind "Dependencies") (eq $change.Kind "Security") }}
|
||||
{{- $changes := splitList " " $change.Custom.PR }}
|
||||
{{- range $issueNbr := $changes }}
|
||||
{{- $changeLink := "[#nbr](https://github.com/dbt-labs/dbt-core/pull/nbr)" | replace "nbr" $issueNbr }}
|
||||
{{- $IssueList = append $IssueList $changeLink }}
|
||||
{{- end -}}
|
||||
{{- else }}
|
||||
{{- $changes := splitList " " $change.Custom.Issue }}
|
||||
{{- range $issueNbr := $changes }}
|
||||
{{- $changeLink := "[#nbr](https://github.com/dbt-labs/dbt-core/issues/nbr)" | replace "nbr" $issueNbr }}
|
||||
{{- $IssueList = append $IssueList $changeLink }}
|
||||
{{- end -}}
|
||||
{{- end }}
|
||||
{{- /* check if this contributor has other changes associated with them already */}}
|
||||
{{- if hasKey $contributorDict $author }}
|
||||
{{- $contributionList := get $contributorDict $author }}
|
||||
{{- $contributionList = concat $contributionList $IssueList }}
|
||||
{{- $contributorDict := set $contributorDict $author $contributionList }}
|
||||
{{- else }}
|
||||
{{- $contributionList := $IssueList }}
|
||||
{{- $contributorDict := set $contributorDict $author $contributionList }}
|
||||
{{- end }}
|
||||
{{- end}}
|
||||
{{- end}}
|
||||
{{- end }}
|
||||
{{- /* no indentation here for formatting so the final markdown doesn't have unneeded indentations */}}
|
||||
|
||||
4
.github/workflows/bot-changelog.yml
vendored
4
.github/workflows/bot-changelog.yml
vendored
@@ -40,7 +40,7 @@ jobs:
|
||||
matrix:
|
||||
include:
|
||||
- label: "dependencies"
|
||||
changie_kind: "Dependency"
|
||||
changie_kind: "Dependencies"
|
||||
- label: "snyk"
|
||||
changie_kind: "Security"
|
||||
runs-on: ubuntu-latest
|
||||
@@ -58,4 +58,4 @@ jobs:
|
||||
commit_message: "Add automated changelog yaml from template for bot PR"
|
||||
changie_kind: ${{ matrix.changie_kind }}
|
||||
label: ${{ matrix.label }}
|
||||
custom_changelog_string: "custom:\n Author: ${{ github.event.pull_request.user.login }}\n Issue: 4904\n PR: ${{ github.event.pull_request.number }}"
|
||||
custom_changelog_string: "custom:\n Author: ${{ github.event.pull_request.user.login }}\n PR: ${{ github.event.pull_request.number }}"
|
||||
|
||||
3
.github/workflows/jira-transition.yml
vendored
3
.github/workflows/jira-transition.yml
vendored
@@ -15,6 +15,9 @@ on:
|
||||
issues:
|
||||
types: [closed, deleted, reopened]
|
||||
|
||||
# no special access is needed
|
||||
permissions: read-all
|
||||
|
||||
jobs:
|
||||
call-label-action:
|
||||
uses: dbt-labs/jira-actions/.github/workflows/jira-transition.yml@main
|
||||
|
||||
14
.github/workflows/main.yml
vendored
14
.github/workflows/main.yml
vendored
@@ -45,7 +45,9 @@ jobs:
|
||||
uses: actions/checkout@v2
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v2
|
||||
uses: actions/setup-python@v4.3.0
|
||||
with:
|
||||
python-version: '3.8'
|
||||
|
||||
- name: Install python dependencies
|
||||
run: |
|
||||
@@ -82,7 +84,7 @@ jobs:
|
||||
uses: actions/checkout@v2
|
||||
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
uses: actions/setup-python@v2
|
||||
uses: actions/setup-python@v4.3.0
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
|
||||
@@ -117,7 +119,7 @@ jobs:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
python-version: ["3.7", "3.8", "3.9", "3.10"]
|
||||
os: [ubuntu-latest]
|
||||
os: [ubuntu-20.04]
|
||||
include:
|
||||
- python-version: 3.8
|
||||
os: windows-latest
|
||||
@@ -137,7 +139,7 @@ jobs:
|
||||
uses: actions/checkout@v2
|
||||
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
uses: actions/setup-python@v2
|
||||
uses: actions/setup-python@v4.3.0
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
|
||||
@@ -190,9 +192,9 @@ jobs:
|
||||
uses: actions/checkout@v2
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v2
|
||||
uses: actions/setup-python@v4.3.0
|
||||
with:
|
||||
python-version: 3.8
|
||||
python-version: '3.8'
|
||||
|
||||
- name: Install python dependencies
|
||||
run: |
|
||||
|
||||
109
.github/workflows/nightly-release.yml
vendored
Normal file
109
.github/workflows/nightly-release.yml
vendored
Normal file
@@ -0,0 +1,109 @@
|
||||
# **what?**
|
||||
# Nightly releases to GitHub and PyPI. This workflow produces the following outcome:
|
||||
# - generate and validate data for night release (commit SHA, version number, release branch);
|
||||
# - pass data to release workflow;
|
||||
# - night release will be pushed to GitHub as a draft release;
|
||||
# - night build will be pushed to test PyPI;
|
||||
#
|
||||
# **why?**
|
||||
# Ensure an automated and tested release process for nightly builds
|
||||
#
|
||||
# **when?**
|
||||
# This workflow runs on schedule or can be run manually on demand.
|
||||
|
||||
name: Nightly Test Release to GitHub and PyPI
|
||||
|
||||
on:
|
||||
workflow_dispatch: # for manual triggering
|
||||
schedule:
|
||||
- cron: 0 9 * * *
|
||||
|
||||
permissions:
|
||||
contents: write # this is the permission that allows creating a new release
|
||||
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
|
||||
env:
|
||||
RELEASE_BRANCH: "main"
|
||||
|
||||
jobs:
|
||||
aggregate-release-data:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
outputs:
|
||||
commit_sha: ${{ steps.resolve-commit-sha.outputs.release_commit }}
|
||||
version_number: ${{ steps.nightly-release-version.outputs.number }}
|
||||
release_branch: ${{ steps.release-branch.outputs.name }}
|
||||
|
||||
steps:
|
||||
- name: "Checkout ${{ github.repository }} Branch ${{ env.RELEASE_BRANCH }}"
|
||||
uses: actions/checkout@v3
|
||||
with:
|
||||
ref: ${{ env.RELEASE_BRANCH }}
|
||||
|
||||
- name: "Resolve Commit To Release"
|
||||
id: resolve-commit-sha
|
||||
run: |
|
||||
commit_sha=$(git rev-parse HEAD)
|
||||
echo "release_commit=$commit_sha" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: "Get Current Version Number"
|
||||
id: version-number-sources
|
||||
run: |
|
||||
current_version=`awk -F"current_version = " '{print $2}' .bumpversion.cfg | tr '\n' ' '`
|
||||
echo "current_version=$current_version" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: "Audit Version And Parse Into Parts"
|
||||
id: semver
|
||||
uses: dbt-labs/actions/parse-semver@v1.1.0
|
||||
with:
|
||||
version: ${{ steps.version-number-sources.outputs.current_version }}
|
||||
|
||||
- name: "Get Current Date"
|
||||
id: current-date
|
||||
run: echo "date=$(date +'%m%d%Y')" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: "Generate Nightly Release Version Number"
|
||||
id: nightly-release-version
|
||||
run: |
|
||||
number="${{ steps.semver.outputs.version }}.dev${{ steps.current-date.outputs.date }}+nightly"
|
||||
echo "number=$number" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: "Audit Nightly Release Version And Parse Into Parts"
|
||||
uses: dbt-labs/actions/parse-semver@v1.1.0
|
||||
with:
|
||||
version: ${{ steps.nightly-release-version.outputs.number }}
|
||||
|
||||
- name: "Set Release Branch"
|
||||
id: release-branch
|
||||
run: |
|
||||
echo "name=${{ env.RELEASE_BRANCH }}" >> $GITHUB_OUTPUT
|
||||
|
||||
log-outputs-aggregate-release-data:
|
||||
runs-on: ubuntu-latest
|
||||
needs: [aggregate-release-data]
|
||||
|
||||
steps:
|
||||
- name: "[DEBUG] Log Outputs"
|
||||
run: |
|
||||
echo commit_sha : ${{ needs.aggregate-release-data.outputs.commit_sha }}
|
||||
echo version_number: ${{ needs.aggregate-release-data.outputs.version_number }}
|
||||
echo release_branch: ${{ needs.aggregate-release-data.outputs.release_branch }}
|
||||
|
||||
release-github-pypi:
|
||||
needs: [aggregate-release-data]
|
||||
|
||||
uses: ./.github/workflows/release.yml
|
||||
with:
|
||||
sha: ${{ needs.aggregate-release-data.outputs.commit_sha }}
|
||||
target_branch: ${{ needs.aggregate-release-data.outputs.release-branch }}
|
||||
version_number: ${{ needs.aggregate-release-data.outputs.version_number }}
|
||||
build_script_path: "scripts/build-dist.sh"
|
||||
env_setup_script_path: "scripts/env-setup.sh"
|
||||
s3_bucket_name: "core-team-artifacts"
|
||||
package_test_command: "dbt --version"
|
||||
test_run: true
|
||||
nightly_release: true
|
||||
secrets: inherit
|
||||
343
.github/workflows/release.yml
vendored
343
.github/workflows/release.yml
vendored
@@ -1,199 +1,230 @@
|
||||
# **what?**
|
||||
# Take the given commit, run unit tests specifically on that sha, build and
|
||||
# package it, and then release to GitHub and PyPi with that specific build
|
||||
|
||||
# Release workflow provides the following steps:
|
||||
# - checkout the given commit;
|
||||
# - validate version in sources and changelog file for given version;
|
||||
# - bump the version and generate a changelog if needed;
|
||||
# - merge all changes to the target branch if needed;
|
||||
# - run unit and integration tests against given commit;
|
||||
# - build and package that SHA;
|
||||
# - release it to GitHub and PyPI with that specific build;
|
||||
#
|
||||
# **why?**
|
||||
# Ensure an automated and tested release process
|
||||
|
||||
#
|
||||
# **when?**
|
||||
# This will only run manually with a given sha and version
|
||||
# This workflow can be run manually on demand or can be called by other workflows
|
||||
|
||||
name: Release to GitHub and PyPi
|
||||
name: Release to GitHub and PyPI
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
sha:
|
||||
description: 'The last commit sha in the release'
|
||||
required: true
|
||||
description: "The last commit sha in the release"
|
||||
type: string
|
||||
required: true
|
||||
target_branch:
|
||||
description: "The branch to release from"
|
||||
type: string
|
||||
required: true
|
||||
version_number:
|
||||
description: 'The release version number (i.e. 1.0.0b1)'
|
||||
required: true
|
||||
description: "The release version number (i.e. 1.0.0b1)"
|
||||
type: string
|
||||
required: true
|
||||
build_script_path:
|
||||
description: "Build script path"
|
||||
type: string
|
||||
default: "scripts/build-dist.sh"
|
||||
required: true
|
||||
env_setup_script_path:
|
||||
description: "Environment setup script path"
|
||||
type: string
|
||||
default: "scripts/env-setup.sh"
|
||||
required: false
|
||||
s3_bucket_name:
|
||||
description: "AWS S3 bucket name"
|
||||
type: string
|
||||
default: "core-team-artifacts"
|
||||
required: true
|
||||
package_test_command:
|
||||
description: "Package test command"
|
||||
type: string
|
||||
default: "dbt --version"
|
||||
required: true
|
||||
test_run:
|
||||
description: "Test run (Publish release as draft)"
|
||||
type: boolean
|
||||
default: true
|
||||
required: false
|
||||
nightly_release:
|
||||
description: "Nightly release to dev environment"
|
||||
type: boolean
|
||||
default: false
|
||||
required: false
|
||||
workflow_call:
|
||||
inputs:
|
||||
sha:
|
||||
description: "The last commit sha in the release"
|
||||
type: string
|
||||
required: true
|
||||
target_branch:
|
||||
description: "The branch to release from"
|
||||
type: string
|
||||
required: true
|
||||
version_number:
|
||||
description: "The release version number (i.e. 1.0.0b1)"
|
||||
type: string
|
||||
required: true
|
||||
build_script_path:
|
||||
description: "Build script path"
|
||||
type: string
|
||||
default: "scripts/build-dist.sh"
|
||||
required: true
|
||||
env_setup_script_path:
|
||||
description: "Environment setup script path"
|
||||
type: string
|
||||
default: "scripts/env-setup.sh"
|
||||
required: false
|
||||
s3_bucket_name:
|
||||
description: "AWS S3 bucket name"
|
||||
type: string
|
||||
default: "core-team-artifacts"
|
||||
required: true
|
||||
package_test_command:
|
||||
description: "Package test command"
|
||||
type: string
|
||||
default: "dbt --version"
|
||||
required: true
|
||||
test_run:
|
||||
description: "Test run (Publish release as draft)"
|
||||
type: boolean
|
||||
default: true
|
||||
required: false
|
||||
nightly_release:
|
||||
description: "Nightly release to dev environment"
|
||||
type: boolean
|
||||
default: false
|
||||
required: false
|
||||
|
||||
permissions:
|
||||
contents: write # this is the permission that allows creating a new release
|
||||
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
|
||||
jobs:
|
||||
unit:
|
||||
name: Unit test
|
||||
|
||||
log-inputs:
|
||||
name: Log Inputs
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
env:
|
||||
TOXENV: "unit"
|
||||
|
||||
steps:
|
||||
- name: Check out the repository
|
||||
uses: actions/checkout@v2
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: ${{ github.event.inputs.sha }}
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v2
|
||||
with:
|
||||
python-version: 3.8
|
||||
|
||||
- name: Install python dependencies
|
||||
- name: "[DEBUG] Print Variables"
|
||||
run: |
|
||||
pip install --user --upgrade pip
|
||||
pip install tox
|
||||
pip --version
|
||||
tox --version
|
||||
echo The last commit sha in the release: ${{ inputs.sha }}
|
||||
echo The branch to release from: ${{ inputs.target_branch }}
|
||||
echo The release version number: ${{ inputs.version_number }}
|
||||
echo Build script path: ${{ inputs.build_script_path }}
|
||||
echo Environment setup script path: ${{ inputs.env_setup_script_path }}
|
||||
echo AWS S3 bucket name: ${{ inputs.s3_bucket_name }}
|
||||
echo Package test command: ${{ inputs.package_test_command }}
|
||||
echo Test run: ${{ inputs.test_run }}
|
||||
echo Nightly release: ${{ inputs.nightly_release }}
|
||||
|
||||
- name: Run tox
|
||||
run: tox
|
||||
bump-version-generate-changelog:
|
||||
name: Bump package version, Generate changelog
|
||||
|
||||
build:
|
||||
name: build packages
|
||||
uses: dbt-labs/dbt-release/.github/workflows/release-prep.yml@main
|
||||
|
||||
with:
|
||||
sha: ${{ inputs.sha }}
|
||||
version_number: ${{ inputs.version_number }}
|
||||
target_branch: ${{ inputs.target_branch }}
|
||||
env_setup_script_path: ${{ inputs.env_setup_script_path }}
|
||||
test_run: ${{ inputs.test_run }}
|
||||
nightly_release: ${{ inputs.nightly_release }}
|
||||
|
||||
secrets:
|
||||
FISHTOWN_BOT_PAT: ${{ secrets.FISHTOWN_BOT_PAT }}
|
||||
|
||||
log-outputs-bump-version-generate-changelog:
|
||||
name: "[Log output] Bump package version, Generate changelog"
|
||||
if: ${{ !failure() && !cancelled() }}
|
||||
|
||||
needs: [bump-version-generate-changelog]
|
||||
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Check out the repository
|
||||
uses: actions/checkout@v2
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: ${{ github.event.inputs.sha }}
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v2
|
||||
with:
|
||||
python-version: 3.8
|
||||
|
||||
- name: Install python dependencies
|
||||
- name: Print variables
|
||||
run: |
|
||||
pip install --user --upgrade pip
|
||||
pip install --upgrade setuptools wheel twine check-wheel-contents
|
||||
pip --version
|
||||
echo Final SHA : ${{ needs.bump-version-generate-changelog.outputs.final_sha }}
|
||||
echo Changelog path: ${{ needs.bump-version-generate-changelog.outputs.changelog_path }}
|
||||
|
||||
- name: Build distributions
|
||||
run: ./scripts/build-dist.sh
|
||||
build-test-package:
|
||||
name: Build, Test, Package
|
||||
if: ${{ !failure() && !cancelled() }}
|
||||
needs: [bump-version-generate-changelog]
|
||||
|
||||
- name: Show distributions
|
||||
run: ls -lh dist/
|
||||
uses: dbt-labs/dbt-release/.github/workflows/build.yml@main
|
||||
|
||||
- name: Check distribution descriptions
|
||||
run: |
|
||||
twine check dist/*
|
||||
with:
|
||||
sha: ${{ needs.bump-version-generate-changelog.outputs.final_sha }}
|
||||
version_number: ${{ inputs.version_number }}
|
||||
changelog_path: ${{ needs.bump-version-generate-changelog.outputs.changelog_path }}
|
||||
build_script_path: ${{ inputs.build_script_path }}
|
||||
s3_bucket_name: ${{ inputs.s3_bucket_name }}
|
||||
package_test_command: ${{ inputs.package_test_command }}
|
||||
test_run: ${{ inputs.test_run }}
|
||||
nightly_release: ${{ inputs.nightly_release }}
|
||||
|
||||
- name: Check wheel contents
|
||||
run: |
|
||||
check-wheel-contents dist/*.whl --ignore W007,W008
|
||||
|
||||
- uses: actions/upload-artifact@v2
|
||||
with:
|
||||
name: dist
|
||||
path: |
|
||||
dist/
|
||||
!dist/dbt-${{github.event.inputs.version_number}}.tar.gz
|
||||
|
||||
test-build:
|
||||
name: verify packages
|
||||
|
||||
needs: [build, unit]
|
||||
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v2
|
||||
with:
|
||||
python-version: 3.8
|
||||
|
||||
- name: Install python dependencies
|
||||
run: |
|
||||
pip install --user --upgrade pip
|
||||
pip install --upgrade wheel
|
||||
pip --version
|
||||
|
||||
- uses: actions/download-artifact@v2
|
||||
with:
|
||||
name: dist
|
||||
path: dist/
|
||||
|
||||
- name: Show distributions
|
||||
run: ls -lh dist/
|
||||
|
||||
- name: Install wheel distributions
|
||||
run: |
|
||||
find ./dist/*.whl -maxdepth 1 -type f | xargs pip install --force-reinstall --find-links=dist/
|
||||
|
||||
- name: Check wheel distributions
|
||||
run: |
|
||||
dbt --version
|
||||
|
||||
- name: Install source distributions
|
||||
run: |
|
||||
find ./dist/*.gz -maxdepth 1 -type f | xargs pip install --force-reinstall --find-links=dist/
|
||||
|
||||
- name: Check source distributions
|
||||
run: |
|
||||
dbt --version
|
||||
secrets:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
|
||||
github-release:
|
||||
name: GitHub Release
|
||||
if: ${{ !failure() && !cancelled() }}
|
||||
|
||||
needs: test-build
|
||||
needs: [bump-version-generate-changelog, build-test-package]
|
||||
|
||||
runs-on: ubuntu-latest
|
||||
uses: dbt-labs/dbt-release/.github/workflows/github-release.yml@main
|
||||
|
||||
steps:
|
||||
- uses: actions/download-artifact@v2
|
||||
with:
|
||||
name: dist
|
||||
path: '.'
|
||||
|
||||
# Need to set an output variable because env variables can't be taken as input
|
||||
# This is needed for the next step with releasing to GitHub
|
||||
- name: Find release type
|
||||
id: release_type
|
||||
env:
|
||||
IS_PRERELEASE: ${{ contains(github.event.inputs.version_number, 'rc') || contains(github.event.inputs.version_number, 'b') }}
|
||||
run: |
|
||||
echo ::set-output name=isPrerelease::$IS_PRERELEASE
|
||||
|
||||
- name: Creating GitHub Release
|
||||
uses: softprops/action-gh-release@v1
|
||||
with:
|
||||
name: dbt-core v${{github.event.inputs.version_number}}
|
||||
tag_name: v${{github.event.inputs.version_number}}
|
||||
prerelease: ${{ steps.release_type.outputs.isPrerelease }}
|
||||
target_commitish: ${{github.event.inputs.sha}}
|
||||
body: |
|
||||
[Release notes](https://github.com/dbt-labs/dbt-core/blob/main/CHANGELOG.md)
|
||||
files: |
|
||||
dbt_postgres-${{github.event.inputs.version_number}}-py3-none-any.whl
|
||||
dbt_core-${{github.event.inputs.version_number}}-py3-none-any.whl
|
||||
dbt-postgres-${{github.event.inputs.version_number}}.tar.gz
|
||||
dbt-core-${{github.event.inputs.version_number}}.tar.gz
|
||||
with:
|
||||
sha: ${{ needs.bump-version-generate-changelog.outputs.final_sha }}
|
||||
version_number: ${{ inputs.version_number }}
|
||||
changelog_path: ${{ needs.bump-version-generate-changelog.outputs.changelog_path }}
|
||||
test_run: ${{ inputs.test_run }}
|
||||
|
||||
pypi-release:
|
||||
name: Pypi release
|
||||
name: PyPI Release
|
||||
|
||||
runs-on: ubuntu-latest
|
||||
needs: [github-release]
|
||||
|
||||
needs: github-release
|
||||
uses: dbt-labs/dbt-release/.github/workflows/pypi-release.yml@main
|
||||
|
||||
environment: PypiProd
|
||||
steps:
|
||||
- uses: actions/download-artifact@v2
|
||||
with:
|
||||
name: dist
|
||||
path: 'dist'
|
||||
with:
|
||||
version_number: ${{ inputs.version_number }}
|
||||
test_run: ${{ inputs.test_run }}
|
||||
|
||||
- name: Publish distribution to PyPI
|
||||
uses: pypa/gh-action-pypi-publish@v1.4.2
|
||||
with:
|
||||
password: ${{ secrets.PYPI_API_TOKEN }}
|
||||
secrets:
|
||||
PYPI_API_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
|
||||
TEST_PYPI_API_TOKEN: ${{ secrets.TEST_PYPI_API_TOKEN }}
|
||||
|
||||
slack-notification:
|
||||
name: Slack Notification
|
||||
if: ${{ failure() && (!inputs.test_run || inputs.nightly_release) }}
|
||||
|
||||
needs:
|
||||
[
|
||||
bump-version-generate-changelog,
|
||||
build-test-package,
|
||||
github-release,
|
||||
pypi-release,
|
||||
]
|
||||
|
||||
uses: dbt-labs/dbt-release/.github/workflows/slack-post-notification.yml@main
|
||||
with:
|
||||
status: "failure"
|
||||
|
||||
secrets:
|
||||
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_DEV_CORE_ALERTS }}
|
||||
|
||||
3
.github/workflows/schema-check.yml
vendored
3
.github/workflows/schema-check.yml
vendored
@@ -21,6 +21,9 @@ on:
|
||||
- "*.latest"
|
||||
- "releases/*"
|
||||
|
||||
# no special access is needed
|
||||
permissions: read-all
|
||||
|
||||
env:
|
||||
LATEST_SCHEMA_PATH: ${{ github.workspace }}/new_schemas
|
||||
SCHEMA_DIFF_ARTIFACT: ${{ github.workspace }}//schema_schanges.txt
|
||||
|
||||
4
.github/workflows/stale.yml
vendored
4
.github/workflows/stale.yml
vendored
@@ -3,6 +3,10 @@ on:
|
||||
schedule:
|
||||
- cron: "30 1 * * *"
|
||||
|
||||
permissions:
|
||||
issues: write
|
||||
pull-requests: write
|
||||
|
||||
jobs:
|
||||
stale:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
@@ -22,7 +22,7 @@ jobs:
|
||||
# run the performance measurements on the current or default branch
|
||||
test-schema:
|
||||
name: Test Log Schema
|
||||
runs-on: ubuntu-latest
|
||||
runs-on: ubuntu-20.04
|
||||
env:
|
||||
# turns warnings into errors
|
||||
RUSTFLAGS: "-D warnings"
|
||||
|
||||
38
.github/workflows/version-bump.yml
vendored
38
.github/workflows/version-bump.yml
vendored
@@ -20,6 +20,10 @@ on:
|
||||
description: 'The version number to bump to (ex. 1.2.0, 1.3.0b1)'
|
||||
required: true
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
|
||||
jobs:
|
||||
bump:
|
||||
runs-on: ubuntu-latest
|
||||
@@ -42,6 +46,16 @@ jobs:
|
||||
source env/bin/activate
|
||||
pip install --upgrade pip
|
||||
|
||||
- name: Add Homebrew to PATH
|
||||
run: |
|
||||
echo "/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin" >> $GITHUB_PATH
|
||||
|
||||
- name: Install Homebrew packages
|
||||
run: |
|
||||
brew install pre-commit
|
||||
brew tap miniscruff/changie https://github.com/miniscruff/changie
|
||||
brew install changie
|
||||
|
||||
- name: Audit Version and Parse Into Parts
|
||||
id: semver
|
||||
uses: dbt-labs/actions/parse-semver@v1
|
||||
@@ -66,18 +80,8 @@ jobs:
|
||||
env/bin/bumpversion --allow-dirty --new-version ${{ github.event.inputs.version_number }} major
|
||||
git status
|
||||
|
||||
# this step will fail on whitespace errors but also correct them
|
||||
- name: Format bumpversion file
|
||||
continue-on-error: true
|
||||
run: |
|
||||
brew install pre-commit
|
||||
pre-commit run trailing-whitespace --files .bumpversion.cfg
|
||||
git status
|
||||
|
||||
- name: Run changie
|
||||
run: |
|
||||
brew tap miniscruff/changie https://github.com/miniscruff/changie
|
||||
brew install changie
|
||||
if [[ ${{ steps.semver.outputs.is-pre-release }} -eq 1 ]]
|
||||
then
|
||||
changie batch ${{ steps.semver.outputs.base-version }} --move-dir '${{ steps.semver.outputs.base-version }}' --prerelease '${{ steps.semver.outputs.pre-release }}'
|
||||
@@ -87,6 +91,20 @@ jobs:
|
||||
changie merge
|
||||
git status
|
||||
|
||||
# this step will fail on whitespace errors but also correct them
|
||||
- name: Remove trailing whitespace
|
||||
continue-on-error: true
|
||||
run: |
|
||||
pre-commit run trailing-whitespace --files .bumpversion.cfg CHANGELOG.md .changes/*
|
||||
git status
|
||||
|
||||
# this step will fail on newline errors but also correct them
|
||||
- name: Removing extra newlines
|
||||
continue-on-error: true
|
||||
run: |
|
||||
pre-commit run end-of-file-fixer --files .bumpversion.cfg CHANGELOG.md .changes/*
|
||||
git status
|
||||
|
||||
- name: Commit version bump to branch
|
||||
uses: EndBug/add-and-commit@v7
|
||||
with:
|
||||
|
||||
1
.gitignore
vendored
1
.gitignore
vendored
@@ -95,6 +95,7 @@ venv/
|
||||
|
||||
# vscode
|
||||
.vscode/
|
||||
*.code-workspace
|
||||
|
||||
# poetry
|
||||
pyproject.toml
|
||||
|
||||
@@ -35,7 +35,7 @@ repos:
|
||||
- "--target-version=py38"
|
||||
- "--check"
|
||||
- "--diff"
|
||||
- repo: https://gitlab.com/pycqa/flake8
|
||||
- repo: https://github.com/pycqa/flake8
|
||||
rev: 4.0.1
|
||||
hooks:
|
||||
- id: flake8
|
||||
|
||||
173
CHANGELOG.md
173
CHANGELOG.md
@@ -5,95 +5,182 @@
|
||||
- "Breaking changes" listed under a version may require action from end users or external maintainers when upgrading to that version.
|
||||
- Do not edit this file directly. This file is auto-generated using [changie](https://github.com/miniscruff/changie). For details on how to document a change, see [the contributing guide](https://github.com/dbt-labs/dbt-core/blob/main/CONTRIBUTING.md#adding-changelog-entry)
|
||||
|
||||
## dbt-core 1.3.0-b2 - August 29, 2022
|
||||
## dbt-core 1.3.4 - April 19, 2023
|
||||
|
||||
### Fixes
|
||||
|
||||
- Improved failed event serialization handling and associated tests ([#7113](https://github.com/dbt-labs/dbt-core/issues/7113), [#7108](https://github.com/dbt-labs/dbt-core/issues/7108), [#6568](https://github.com/dbt-labs/dbt-core/issues/6568))
|
||||
|
||||
### Contributors
|
||||
- [@QMalcolm](https://github.com/QMalcolm) ([#7113](https://github.com/dbt-labs/dbt-core/issues/7113), [#7108](https://github.com/dbt-labs/dbt-core/issues/7108), [#6568](https://github.com/dbt-labs/dbt-core/issues/6568))
|
||||
|
||||
|
||||
## dbt-core 1.3.3 - February 28, 2023
|
||||
|
||||
### Fixes
|
||||
|
||||
- add pytz dependency ([#7077](https://github.com/dbt-labs/dbt-core/issues/7077))
|
||||
|
||||
### Contributors
|
||||
- [@sdebruyn](https://github.com/sdebruyn) ([#7077](https://github.com/dbt-labs/dbt-core/issues/7077))
|
||||
|
||||
## dbt-core 1.3.2 - January 04, 2023
|
||||
|
||||
### Fixes
|
||||
|
||||
- Bug when partial parsing with an empty schema file ([#4850](https://github.com/dbt-labs/dbt-core/issues/4850), [#<no value>](https://github.com/dbt-labs/dbt-core/pull/<no value>))
|
||||
|
||||
## dbt-core 1.3.1 - November 16, 2022
|
||||
### Features
|
||||
- This pulls the profile name from args when constructing a RuntimeConfig in lib.py, enabling the dbt-server to override the value that's in the dbt_project.yml ([#6201](https://github.com/dbt-labs/dbt-core/issues/6201), [#6202](https://github.com/dbt-labs/dbt-core/pull/6202))
|
||||
### Docs
|
||||
- ([dbt-docs/#5880](https://github.com/dbt-labs/dbt-docs/issues/5880), [dbt-docs/#324](https://github.com/dbt-labs/dbt-docs/pull/324))
|
||||
- Fix rendering of sample code for metrics ([dbt-docs/#323](https://github.com/dbt-labs/dbt-docs/issues/323), [dbt-docs/#346](https://github.com/dbt-labs/dbt-docs/pull/346))
|
||||
|
||||
### Contributors
|
||||
- [@paulbenschmidt](https://github.com/paulbenschmidt) ([dbt-docs/#324](https://github.com/dbt-labs/dbt-docs/pull/324))
|
||||
- [@racheldaniel](https://github.com/racheldaniel) ([#6202](https://github.com/dbt-labs/dbt-core/pull/6202))
|
||||
|
||||
## dbt-core 1.3.0 - October 12, 2022
|
||||
|
||||
### Breaking Changes
|
||||
|
||||
- Renaming Metric Spec Attributes ([#5774](https://github.com/dbt-labs/dbt-core/issues/5774), [#5775](https://github.com/dbt-labs/dbt-core/pull/5775))
|
||||
|
||||
### Features
|
||||
|
||||
- Add `--defer` flag to dbt compile & dbt docs generate ([#4110](https://github.com/dbt-labs/dbt-core/issues/4110), [#4514](https://github.com/dbt-labs/dbt-core/pull/4514))
|
||||
- Python model inital version ([#5261](https://github.com/dbt-labs/dbt-core/issues/5261), [#5421](https://github.com/dbt-labs/dbt-core/pull/5421))
|
||||
- allows user to include the file extension for .py models in the dbt run -m command. ([#5289](https://github.com/dbt-labs/dbt-core/issues/5289), [#5295](https://github.com/dbt-labs/dbt-core/pull/5295))
|
||||
- Incremental materialization refactor and cleanup ([#5245](https://github.com/dbt-labs/dbt-core/issues/5245), [#5359](https://github.com/dbt-labs/dbt-core/pull/5359))
|
||||
- Python models can support incremental logic ([#0](https://github.com/dbt-labs/dbt-core/issues/0), [#35](https://github.com/dbt-labs/dbt-core/pull/35))
|
||||
- Add reusable function for retrying adapter connections. Utilize said function to add retries for Postgres (and Redshift). ([#5022](https://github.com/dbt-labs/dbt-core/issues/5022), [#5432](https://github.com/dbt-labs/dbt-core/pull/5432))
|
||||
- merge_exclude_columns for incremental materialization ([#5260](https://github.com/dbt-labs/dbt-core/issues/5260), [#5457](https://github.com/dbt-labs/dbt-core/pull/5457))
|
||||
- add exponential backoff to connection retries on Postgres (and Redshift) ([#5502](https://github.com/dbt-labs/dbt-core/issues/5502), [#5503](https://github.com/dbt-labs/dbt-core/pull/5503))
|
||||
- use MethodName.File when value ends with .csv ([#5578](https://github.com/dbt-labs/dbt-core/issues/5578), [#5581](https://github.com/dbt-labs/dbt-core/pull/5581))
|
||||
- Make `docs` configurable in `dbt_project.yml` and add a `node_color` attribute to change the color of nodes in the DAG ([#5333](https://github.com/dbt-labs/dbt-core/issues/5333), [#5397](https://github.com/dbt-labs/dbt-core/pull/5397))
|
||||
- Adding ResolvedMetricReference helper functions and tests ([#5567](https://github.com/dbt-labs/dbt-core/issues/5567), [#5607](https://github.com/dbt-labs/dbt-core/pull/5607))
|
||||
- Check dbt-core version requirements when installing Hub packages ([#5648](https://github.com/dbt-labs/dbt-core/issues/5648), [#5651](https://github.com/dbt-labs/dbt-core/pull/5651))
|
||||
- Search current working directory for `profiles.yml` ([#5411](https://github.com/dbt-labs/dbt-core/issues/5411), [#5717](https://github.com/dbt-labs/dbt-core/pull/5717))
|
||||
- Adding the `window` parameter to the metric spec. ([#5721](https://github.com/dbt-labs/dbt-core/issues/5721), [#5722](https://github.com/dbt-labs/dbt-core/pull/5722))
|
||||
- Add invocation args dict to ProviderContext class ([#5524](https://github.com/dbt-labs/dbt-core/issues/5524), [#5782](https://github.com/dbt-labs/dbt-core/pull/5782))
|
||||
- Adds new cli framework ([#5526](https://github.com/dbt-labs/dbt-core/issues/5526), [#5647](https://github.com/dbt-labs/dbt-core/pull/5647))
|
||||
- Flags work with new Click CLI ([#5529](https://github.com/dbt-labs/dbt-core/issues/5529), [#5790](https://github.com/dbt-labs/dbt-core/pull/5790))
|
||||
- Add metadata env method to ProviderContext class ([#5522](https://github.com/dbt-labs/dbt-core/issues/5522), [#5794](https://github.com/dbt-labs/dbt-core/pull/5794))
|
||||
- Array macros ([#5520](https://github.com/dbt-labs/dbt-core/issues/5520), [#5823](https://github.com/dbt-labs/dbt-core/pull/5823))
|
||||
- Add enabled config to exposures and metrics ([#5422](https://github.com/dbt-labs/dbt-core/issues/5422), [#5815](https://github.com/dbt-labs/dbt-core/pull/5815))
|
||||
- Migrate dbt-utils current_timestamp macros into core + adapters ([#5521](https://github.com/dbt-labs/dbt-core/issues/5521), [#5838](https://github.com/dbt-labs/dbt-core/pull/5838))
|
||||
- add -fr flag shorthand ([#5878](https://github.com/dbt-labs/dbt-core/issues/5878), [#5879](https://github.com/dbt-labs/dbt-core/pull/5879))
|
||||
- add type_boolean as a data type macro ([#5739](https://github.com/dbt-labs/dbt-core/issues/5739), [#5875](https://github.com/dbt-labs/dbt-core/pull/5875))
|
||||
- Support .dbtignore in project root to ignore certain files being read by dbt ([#5733](https://github.com/dbt-labs/dbt-core/issues/5733), [#5897](https://github.com/dbt-labs/dbt-core/pull/5897))
|
||||
- This conditionally no-ops warehouse connection at compile depending on an env var, disabling introspection/queries during compilation only. This is a temporary solution to more complex permissions requirements for the semantic layer. ([#5936](https://github.com/dbt-labs/dbt-core/issues/5936), [#5926](https://github.com/dbt-labs/dbt-core/pull/5926))
|
||||
|
||||
### Fixes
|
||||
|
||||
- Remove the default 256 characters limit on postgres character varying type when no limitation is set ([#5238](https://github.com/dbt-labs/dbt-core/issues/5238), [#5292](https://github.com/dbt-labs/dbt-core/pull/5292))
|
||||
- Include schema file config in unrendered_config ([#5338](https://github.com/dbt-labs/dbt-core/issues/5338), [#5344](https://github.com/dbt-labs/dbt-core/pull/5344))
|
||||
- Add context to compilation errors generated while rendering generic test configuration values. ([#5294](https://github.com/dbt-labs/dbt-core/issues/5294), [#5393](https://github.com/dbt-labs/dbt-core/pull/5393))
|
||||
- Resolves #5351 - Do not consider shorter varchar cols as schema changes ([#5351](https://github.com/dbt-labs/dbt-core/issues/5351), [#5395](https://github.com/dbt-labs/dbt-core/pull/5395))
|
||||
- Rename try to strict for more intuitiveness ([#5475](https://github.com/dbt-labs/dbt-core/issues/5475), [#5477](https://github.com/dbt-labs/dbt-core/pull/5477))
|
||||
- on_shchma_change fail verbosity enhancement ([#5504](https://github.com/dbt-labs/dbt-core/issues/5504), [#5505](https://github.com/dbt-labs/dbt-core/pull/5505))
|
||||
- Ignore empty strings passed in as secrets ([#5312](https://github.com/dbt-labs/dbt-core/issues/5312), [#5518](https://github.com/dbt-labs/dbt-core/pull/5518))
|
||||
- Fix handling of top-level exceptions ([#5564](https://github.com/dbt-labs/dbt-core/issues/5564), [#5560](https://github.com/dbt-labs/dbt-core/pull/5560))
|
||||
- Fix error rendering docs block in metrics description ([#5585](https://github.com/dbt-labs/dbt-core/issues/5585), [#5603](https://github.com/dbt-labs/dbt-core/pull/5603))
|
||||
- Extended validations for the project names ([#5379](https://github.com/dbt-labs/dbt-core/issues/5379), [#5620](https://github.com/dbt-labs/dbt-core/pull/5620))
|
||||
- Use sys.exit instead of exit ([#5621](https://github.com/dbt-labs/dbt-core/issues/5621), [#5627](https://github.com/dbt-labs/dbt-core/pull/5627))
|
||||
- Finishing logic upgrade to Redshift for name truncation collisions. ([#5586](https://github.com/dbt-labs/dbt-core/issues/5586), [#5656](https://github.com/dbt-labs/dbt-core/pull/5656))
|
||||
- multiple args for ref and source ([#5634](https://github.com/dbt-labs/dbt-core/issues/5634), [#5635](https://github.com/dbt-labs/dbt-core/pull/5635))
|
||||
- Fix Unexpected behavior when chaining methods on dbt-ref'ed/sourced dataframes ([#5646](https://github.com/dbt-labs/dbt-core/issues/5646), [#5677](https://github.com/dbt-labs/dbt-core/pull/5677))
|
||||
- Fix typos of comments in core/dbt/adapters/ ([#5690](https://github.com/dbt-labs/dbt-core/issues/5690), [#5693](https://github.com/dbt-labs/dbt-core/pull/5693))
|
||||
- Include py.typed in MANIFEST.in. This enables packages that install dbt-core from pypi to use mypy. ([#5703](https://github.com/dbt-labs/dbt-core/issues/5703), [#5703](https://github.com/dbt-labs/dbt-core/pull/5703))
|
||||
- Removal of all .coverage files when using make clean command ([#5633](https://github.com/dbt-labs/dbt-core/issues/5633), [#5759](https://github.com/dbt-labs/dbt-core/pull/5759))
|
||||
- Remove temp files generated by unit tests ([#5631](https://github.com/dbt-labs/dbt-core/issues/5631), [#5749](https://github.com/dbt-labs/dbt-core/pull/5749))
|
||||
- Fix warnings as errors during tests ([#5424](https://github.com/dbt-labs/dbt-core/issues/5424), [#5800](https://github.com/dbt-labs/dbt-core/pull/5800))
|
||||
- Prevent event_history from holding references ([#5848](https://github.com/dbt-labs/dbt-core/issues/5848), [#5858](https://github.com/dbt-labs/dbt-core/pull/5858))
|
||||
- Account for disabled flags on models in schema files more completely ([#3992](https://github.com/dbt-labs/dbt-core/issues/3992), [#5868](https://github.com/dbt-labs/dbt-core/pull/5868))
|
||||
- ConfigSelectorMethod should check for bools ([#5890](https://github.com/dbt-labs/dbt-core/issues/5890), [#5889](https://github.com/dbt-labs/dbt-core/pull/5889))
|
||||
- shorthand for full refresh should be one character ([#5878](https://github.com/dbt-labs/dbt-core/issues/5878), [#5908](https://github.com/dbt-labs/dbt-core/pull/5908))
|
||||
- Fix macro resolution order during static analysis for custom generic tests ([#5720](https://github.com/dbt-labs/dbt-core/issues/5720), [#5907](https://github.com/dbt-labs/dbt-core/pull/5907))
|
||||
- Fix race condition when invoking dbt via lib.py concurrently ([#5919](https://github.com/dbt-labs/dbt-core/issues/5919), [#5921](https://github.com/dbt-labs/dbt-core/pull/5921))
|
||||
- check length of args of python model function before accessing it ([#6041](https://github.com/dbt-labs/dbt-core/issues/6041), [#6042](https://github.com/dbt-labs/dbt-core/pull/6042))
|
||||
|
||||
### Docs
|
||||
|
||||
- Update dependency inline-source from ^6.1.5 to ^7.2.0 ([dbt-docs/#299](https://github.com/dbt-labs/dbt-docs/issues/299), [dbt-docs/#291](https://github.com/dbt-labs/dbt-docs/pull/291))
|
||||
- Update dependency jest from ^26.2.2 to ^28.1.3 ([dbt-docs/#299](https://github.com/dbt-labs/dbt-docs/issues/299), [dbt-docs/#291](https://github.com/dbt-labs/dbt-docs/pull/291))
|
||||
- Update dependency underscore from ^1.9.0 to ^1.13.4 ([dbt-docs/#299](https://github.com/dbt-labs/dbt-docs/issues/299), [dbt-docs/#291](https://github.com/dbt-labs/dbt-docs/pull/291))
|
||||
- Update dependency webpack-cli from ^3.3.12 to ^4.7.0 ([dbt-docs/#299](https://github.com/dbt-labs/dbt-docs/issues/299), [dbt-docs/#291](https://github.com/dbt-labs/dbt-docs/pull/291))
|
||||
- Update dependency webpack-dev-server from ^3.1.11 to ^4.9.3 ([dbt-docs/#299](https://github.com/dbt-labs/dbt-docs/issues/299), [dbt-docs/#291](https://github.com/dbt-labs/dbt-docs/pull/291))
|
||||
- Searches no longer require perfect matches, and instead consider each word individually. `my model` or `model my` will now find `my_model`, without the need for underscores ([dbt-docs/#143](https://github.com/dbt-labs/dbt-docs/issues/143), [dbt-docs/#145](https://github.com/dbt-labs/dbt-docs/pull/145))
|
||||
- Support the renaming of SQL to code happening in dbt-core ([dbt-docs/#299](https://github.com/dbt-labs/dbt-docs/issues/299), [dbt-docs/#292](https://github.com/dbt-labs/dbt-docs/pull/292))
|
||||
- Leverages `docs.node_color` from `dbt-core` to color nodes in the DAG ([dbt-docs/#44](https://github.com/dbt-labs/dbt-docs/issues/44), [dbt-docs/#281](https://github.com/dbt-labs/dbt-docs/pull/281))
|
||||
- Refer to exposures by their label by default. ([dbt-docs/#306](https://github.com/dbt-labs/dbt-docs/issues/306), [dbt-docs/#307](https://github.com/dbt-labs/dbt-docs/pull/307))
|
||||
|
||||
### Under the Hood
|
||||
|
||||
- Added language to tracked fields in run_model event ([#5571](https://github.com/dbt-labs/dbt-core/issues/5571), [#5469](https://github.com/dbt-labs/dbt-core/pull/5469))
|
||||
- Update mashumaro to 3.0.3 ([#4940](https://github.com/dbt-labs/dbt-core/issues/4940), [#5118](https://github.com/dbt-labs/dbt-core/pull/5118))
|
||||
- Add python incremental materialization test ([#0000](https://github.com/dbt-labs/dbt-core/issues/0000), [#5571](https://github.com/dbt-labs/dbt-core/pull/5571))
|
||||
- Save use of default env vars to manifest to enable partial parsing in those cases. ([#5155](https://github.com/dbt-labs/dbt-core/issues/5155), [#5589](https://github.com/dbt-labs/dbt-core/pull/5589))
|
||||
- add more information to log line interop test failures ([#5658](https://github.com/dbt-labs/dbt-core/issues/5658), [#5659](https://github.com/dbt-labs/dbt-core/pull/5659))
|
||||
- Add supported languages to materializations ([#5569](https://github.com/dbt-labs/dbt-core/issues/5569), [#5695](https://github.com/dbt-labs/dbt-core/pull/5695))
|
||||
- Migrate integration test 014 but also fix the snapshot hard delete test's timezone logic and force all integration tests to run flags.set_from_args to force environment variables are accessible to all integration test threads. ([#5760](https://github.com/dbt-labs/dbt-core/issues/5760), [#5760](https://github.com/dbt-labs/dbt-core/pull/5760))
|
||||
- Support dbt-metrics compilation by rebuilding flat_graph ([#5525](https://github.com/dbt-labs/dbt-core/issues/5525), [#5786](https://github.com/dbt-labs/dbt-core/pull/5786))
|
||||
- Reworking the way we define the window attribute of metrics to match freshness tests ([#5722](https://github.com/dbt-labs/dbt-core/issues/5722), [#5793](https://github.com/dbt-labs/dbt-core/pull/5793))
|
||||
- Add PythonJobHelper base class in core and add more type checking ([#5802](https://github.com/dbt-labs/dbt-core/issues/5802), [#5802](https://github.com/dbt-labs/dbt-core/pull/5802))
|
||||
- The link did not go to the anchor directly, now it does ([#5813](https://github.com/dbt-labs/dbt-core/issues/5813), [#5814](https://github.com/dbt-labs/dbt-core/pull/5814))
|
||||
- remove key as reserved keyword from test_bool_or ([#5817](https://github.com/dbt-labs/dbt-core/issues/5817), [#5818](https://github.com/dbt-labs/dbt-core/pull/5818))
|
||||
- Convert default selector tests to pytest ([#5728](https://github.com/dbt-labs/dbt-core/issues/5728), [#5820](https://github.com/dbt-labs/dbt-core/pull/5820))
|
||||
- Compatibiltiy for metric attribute renaming ([#5807](https://github.com/dbt-labs/dbt-core/issues/5807), [#5825](https://github.com/dbt-labs/dbt-core/pull/5825))
|
||||
- remove source quoting setting in adapter tests ([#5836](https://github.com/dbt-labs/dbt-core/issues/5836), [#5839](https://github.com/dbt-labs/dbt-core/pull/5839))
|
||||
- Add name validation for metrics ([#5456](https://github.com/dbt-labs/dbt-core/issues/5456), [#5841](https://github.com/dbt-labs/dbt-core/pull/5841))
|
||||
- Validate exposure name and add label ([#5606](https://github.com/dbt-labs/dbt-core/issues/5606), [#5844](https://github.com/dbt-labs/dbt-core/pull/5844))
|
||||
- Adding validation for metric expression attribute ([#5871](https://github.com/dbt-labs/dbt-core/issues/5871), [#5873](https://github.com/dbt-labs/dbt-core/pull/5873))
|
||||
- Profiling and Adapter Management work with Click CLI ([#5531](https://github.com/dbt-labs/dbt-core/issues/5531), [#5892](https://github.com/dbt-labs/dbt-core/pull/5892))
|
||||
- Reparse references to deleted metric ([#5444](https://github.com/dbt-labs/dbt-core/issues/5444), [#5920](https://github.com/dbt-labs/dbt-core/pull/5920))
|
||||
|
||||
### Dependencies
|
||||
|
||||
- Upgrade to Jinja2==3.1.2 from Jinja2==2.11.3 ([#4748](https://github.com/dbt-labs/dbt-core/issues/4748), [#5465](https://github.com/dbt-labs/dbt-core/pull/5465))
|
||||
- Bump mypy from 0.961 to 0.971 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5495](https://github.com/dbt-labs/dbt-core/pull/5495))
|
||||
- Remove pin for MarkUpSafe from >=0.23,<2.1 ([#5506](https://github.com/dbt-labs/dbt-core/issues/5506), [#5507](https://github.com/dbt-labs/dbt-core/pull/5507))
|
||||
|
||||
### Dependency
|
||||
|
||||
- Bump python from 3.10.5-slim-bullseye to 3.10.6-slim-bullseye in /docker ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5623](https://github.com/dbt-labs/dbt-core/pull/5623))
|
||||
- Bump mashumaro[msgpack] from 3.0.3 to 3.0.4 in /core ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5649](https://github.com/dbt-labs/dbt-core/pull/5649))
|
||||
- Bump black from 22.6.0 to 22.8.0 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5750](https://github.com/dbt-labs/dbt-core/pull/5750))
|
||||
- Bump python from 3.10.6-slim-bullseye to 3.10.7-slim-bullseye in /docker ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5805](https://github.com/dbt-labs/dbt-core/pull/5805))
|
||||
|
||||
### Contributors
|
||||
- [@Goodkat](https://github.com/Goodkat) ([#5581](https://github.com/dbt-labs/dbt-core/pull/5581), [#5620](https://github.com/dbt-labs/dbt-core/pull/5620))
|
||||
- [@Goodkat](https://github.com/Goodkat) ([#5581](https://github.com/dbt-labs/dbt-core/pull/5581), [#5518](https://github.com/dbt-labs/dbt-core/pull/5518), [#5620](https://github.com/dbt-labs/dbt-core/pull/5620))
|
||||
- [@Ilanbenb](https://github.com/Ilanbenb) ([#5505](https://github.com/dbt-labs/dbt-core/pull/5505))
|
||||
- [@b-per](https://github.com/b-per) ([#5397](https://github.com/dbt-labs/dbt-core/pull/5397), [dbt-docs/#281](https://github.com/dbt-labs/dbt-docs/pull/281))
|
||||
- [@callum-mcdata](https://github.com/callum-mcdata) ([#5607](https://github.com/dbt-labs/dbt-core/pull/5607))
|
||||
- [@bbroeksema](https://github.com/bbroeksema) ([#5749](https://github.com/dbt-labs/dbt-core/pull/5749))
|
||||
- [@callum-mcdata](https://github.com/callum-mcdata) ([#5775](https://github.com/dbt-labs/dbt-core/pull/5775), [#5607](https://github.com/dbt-labs/dbt-core/pull/5607), [#5722](https://github.com/dbt-labs/dbt-core/pull/5722), [#5793](https://github.com/dbt-labs/dbt-core/pull/5793), [#5825](https://github.com/dbt-labs/dbt-core/pull/5825), [#5873](https://github.com/dbt-labs/dbt-core/pull/5873))
|
||||
- [@danielcmessias](https://github.com/danielcmessias) ([#5889](https://github.com/dbt-labs/dbt-core/pull/5889))
|
||||
- [@dave-connors-3](https://github.com/dave-connors-3) ([#5457](https://github.com/dbt-labs/dbt-core/pull/5457), [#5879](https://github.com/dbt-labs/dbt-core/pull/5879), [#5908](https://github.com/dbt-labs/dbt-core/pull/5908))
|
||||
- [@dbeatty10](https://github.com/dbeatty10) ([#5717](https://github.com/dbt-labs/dbt-core/pull/5717), [#5823](https://github.com/dbt-labs/dbt-core/pull/5823))
|
||||
- [@drewbanin](https://github.com/drewbanin) ([#5921](https://github.com/dbt-labs/dbt-core/pull/5921), [dbt-docs/#292](https://github.com/dbt-labs/dbt-docs/pull/292))
|
||||
- [@epapineau](https://github.com/epapineau) ([#5395](https://github.com/dbt-labs/dbt-core/pull/5395))
|
||||
- [@graciegoheen](https://github.com/graciegoheen) ([#5823](https://github.com/dbt-labs/dbt-core/pull/5823))
|
||||
- [@jared-rimmer](https://github.com/jared-rimmer) ([#5782](https://github.com/dbt-labs/dbt-core/pull/5782), [#5794](https://github.com/dbt-labs/dbt-core/pull/5794), [#5759](https://github.com/dbt-labs/dbt-core/pull/5759))
|
||||
- [@jeremyyeo](https://github.com/jeremyyeo) ([#5477](https://github.com/dbt-labs/dbt-core/pull/5477))
|
||||
- [@joellabes](https://github.com/joellabes) ([dbt-docs/#145](https://github.com/dbt-labs/dbt-docs/pull/145))
|
||||
- [@jpmmcneill](https://github.com/jpmmcneill) ([#5875](https://github.com/dbt-labs/dbt-core/pull/5875))
|
||||
- [@kadero](https://github.com/kadero) ([#4514](https://github.com/dbt-labs/dbt-core/pull/4514))
|
||||
- [@leoebfolsom](https://github.com/leoebfolsom) ([#5295](https://github.com/dbt-labs/dbt-core/pull/5295))
|
||||
- [@matt-winkler](https://github.com/matt-winkler) ([#5397](https://github.com/dbt-labs/dbt-core/pull/5397), [dbt-docs/#281](https://github.com/dbt-labs/dbt-docs/pull/281))
|
||||
- [@nicholasyager](https://github.com/nicholasyager) ([#5393](https://github.com/dbt-labs/dbt-core/pull/5393))
|
||||
- [@panasenco](https://github.com/panasenco) ([#5703](https://github.com/dbt-labs/dbt-core/pull/5703))
|
||||
- [@racheldaniel](https://github.com/racheldaniel) ([#5926](https://github.com/dbt-labs/dbt-core/pull/5926))
|
||||
- [@sdebruyn](https://github.com/sdebruyn) ([#5814](https://github.com/dbt-labs/dbt-core/pull/5814), [#5818](https://github.com/dbt-labs/dbt-core/pull/5818), [#5839](https://github.com/dbt-labs/dbt-core/pull/5839))
|
||||
- [@shrodingers](https://github.com/shrodingers) ([#5292](https://github.com/dbt-labs/dbt-core/pull/5292))
|
||||
- [@sungchun12](https://github.com/sungchun12) ([#5397](https://github.com/dbt-labs/dbt-core/pull/5397), [dbt-docs/#281](https://github.com/dbt-labs/dbt-docs/pull/281))
|
||||
- [@varun-dc](https://github.com/varun-dc) ([#5627](https://github.com/dbt-labs/dbt-core/pull/5627))
|
||||
|
||||
|
||||
## dbt-core 1.3.0-b1 - July 29, 2022
|
||||
### Features
|
||||
- Python model inital version ([#5261](https://github.com/dbt-labs/dbt-core/issues/5261), [#5421](https://github.com/dbt-labs/dbt-core/pull/5421))
|
||||
- allows user to include the file extension for .py models in the dbt run -m command. ([#5289](https://github.com/dbt-labs/dbt-core/issues/5289), [#5295](https://github.com/dbt-labs/dbt-core/pull/5295))
|
||||
- Incremental materialization refactor and cleanup ([#5245](https://github.com/dbt-labs/dbt-core/issues/5245), [#5359](https://github.com/dbt-labs/dbt-core/pull/5359))
|
||||
- Python models can support incremental logic ([#0](https://github.com/dbt-labs/dbt-core/issues/0), [#35](https://github.com/dbt-labs/dbt-core/pull/35))
|
||||
- Add reusable function for retrying adapter connections. Utilize said function to add retries for Postgres (and Redshift). ([#5022](https://github.com/dbt-labs/dbt-core/issues/5022), [#5432](https://github.com/dbt-labs/dbt-core/pull/5432))
|
||||
- add exponential backoff to connection retries on Postgres (and Redshift) ([#5502](https://github.com/dbt-labs/dbt-core/issues/5502), [#5503](https://github.com/dbt-labs/dbt-core/pull/5503))
|
||||
### Fixes
|
||||
- Add context to compilation errors generated while rendering generic test configuration values. ([#5294](https://github.com/dbt-labs/dbt-core/issues/5294), [#5393](https://github.com/dbt-labs/dbt-core/pull/5393))
|
||||
- Rename try to strict for more intuitiveness ([#5475](https://github.com/dbt-labs/dbt-core/issues/5475), [#5477](https://github.com/dbt-labs/dbt-core/pull/5477))
|
||||
- Ignore empty strings passed in as secrets ([#5312](https://github.com/dbt-labs/dbt-core/issues/5312), [#5518](https://github.com/dbt-labs/dbt-core/pull/5518))
|
||||
- Fix handling of top-level exceptions ([#5564](https://github.com/dbt-labs/dbt-core/issues/5564), [#5560](https://github.com/dbt-labs/dbt-core/pull/5560))
|
||||
### Docs
|
||||
- Update dependency inline-source from ^6.1.5 to ^7.2.0 ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
|
||||
- Update dependency jest from ^26.2.2 to ^28.1.3 ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
|
||||
- Update dependency underscore from ^1.9.0 to ^1.13.4 ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
|
||||
- Update dependency webpack-cli from ^3.3.12 to ^4.7.0 ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
|
||||
- Update dependency webpack-dev-server from ^3.1.11 to ^4.9.3 ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
|
||||
- Searches no longer require perfect matches, and instead consider each word individually. `my model` or `model my` will now find `my_model`, without the need for underscores ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
|
||||
- Support the renaming of SQL to code happening in dbt-core ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
|
||||
### Under the Hood
|
||||
- Added language to tracked fields in run_model event ([#5571](https://github.com/dbt-labs/dbt-core/issues/5571), [#5469](https://github.com/dbt-labs/dbt-core/pull/5469))
|
||||
- Update mashumaro to 3.0.3 ([#4940](https://github.com/dbt-labs/dbt-core/issues/4940), [#5118](https://github.com/dbt-labs/dbt-core/pull/5118))
|
||||
- Add python incremental materialization test ([#0000](https://github.com/dbt-labs/dbt-core/issues/0000), [#5571](https://github.com/dbt-labs/dbt-core/pull/5571))
|
||||
### Dependencies
|
||||
- Upgrade to Jinja2==3.1.2 from Jinja2==2.11.3 ([#4748](https://github.com/dbt-labs/dbt-core/issues/4748), [#5465](https://github.com/dbt-labs/dbt-core/pull/5465))
|
||||
- Bump mypy from 0.961 to 0.971 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5495](https://github.com/dbt-labs/dbt-core/pull/5495))
|
||||
- Remove pin for MarkUpSafe from >=0.23,<2.1 ([#5506](https://github.com/dbt-labs/dbt-core/issues/5506), [#5507](https://github.com/dbt-labs/dbt-core/pull/5507))
|
||||
|
||||
### Contributors
|
||||
- [@Goodkat](https://github.com/Goodkat) ([#5518](https://github.com/dbt-labs/dbt-core/pull/5518))
|
||||
- [@drewbanin](https://github.com/drewbanin) ([#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
|
||||
- [@jeremyyeo](https://github.com/jeremyyeo) ([#5477](https://github.com/dbt-labs/dbt-core/pull/5477))
|
||||
- [@joellabes](https://github.com/joellabes) ([#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
|
||||
- [@leoebfolsom](https://github.com/leoebfolsom) ([#5295](https://github.com/dbt-labs/dbt-core/pull/5295))
|
||||
- [@nicholasyager](https://github.com/nicholasyager) ([#5393](https://github.com/dbt-labs/dbt-core/pull/5393))
|
||||
- [@tomasfarias](https://github.com/tomasfarias) ([#5432](https://github.com/dbt-labs/dbt-core/pull/5432))
|
||||
- [@varun-dc](https://github.com/varun-dc) ([#5627](https://github.com/dbt-labs/dbt-core/pull/5627))
|
||||
- [@yoiki](https://github.com/yoiki) ([#5693](https://github.com/dbt-labs/dbt-core/pull/5693))
|
||||
- [@chamini2](https://github.com/chamini2) ([#6042](https://github.com/dbt-labs/dbt-core/pull/6042))
|
||||
|
||||
## Previous Releases
|
||||
|
||||
|
||||
@@ -201,13 +201,21 @@ Here are some general rules for adding tests:
|
||||
* Sometimes flake8 complains about lines that are actually fine, in which case you can put a comment on the line such as: # noqa or # noqa: ANNN, where ANNN is the error code that flake8 issues.
|
||||
* To collect output for `CProfile`, run dbt with the `-r` option and the name of an output file, i.e. `dbt -r dbt.cprof run`. If you just want to profile parsing, you can do: `dbt -r dbt.cprof parse`. `pip` install `snakeviz` to view the output. Run `snakeviz dbt.cprof` and output will be rendered in a browser window.
|
||||
|
||||
## Adding a CHANGELOG Entry
|
||||
## Adding or modifying a CHANGELOG Entry
|
||||
|
||||
We use [changie](https://changie.dev) to generate `CHANGELOG` entries. **Note:** Do not edit the `CHANGELOG.md` directly. Your modifications will be lost.
|
||||
|
||||
Follow the steps to [install `changie`](https://changie.dev/guide/installation/) for your system.
|
||||
|
||||
Once changie is installed and your PR is created, simply run `changie new` and changie will walk you through the process of creating a changelog entry. Commit the file that's created and your changelog entry is complete!
|
||||
Once changie is installed and your PR is created for a new feature, simply run the following command and changie will walk you through the process of creating a changelog entry:
|
||||
|
||||
```shell
|
||||
changie new
|
||||
```
|
||||
|
||||
Commit the file that's created and your changelog entry is complete!
|
||||
|
||||
If you are contributing to a feature already in progress, you will modify the changie yaml file in dbt/.changes/unreleased/ related to your change. If you need help finding this file, please ask within the discussion for the pull request!
|
||||
|
||||
You don't need to worry about which `dbt-core` version your change will go into. Just create the changelog entry with `changie`, and open your PR against the `main` branch. All merged changes will be included in the next minor version of `dbt-core`. The Core maintainers _may_ choose to "backport" specific changes in order to patch older minor versions. In that case, a maintainer will take care of that backport after merging your PR, before releasing the new version of `dbt-core`.
|
||||
|
||||
|
||||
@@ -12,6 +12,7 @@ class Column:
|
||||
"TIMESTAMP": "TIMESTAMP",
|
||||
"FLOAT": "FLOAT",
|
||||
"INTEGER": "INT",
|
||||
"BOOLEAN": "BOOLEAN",
|
||||
}
|
||||
column: str
|
||||
dtype: str
|
||||
|
||||
@@ -1220,7 +1220,6 @@ class BaseAdapter(metaclass=AdapterMeta):
|
||||
def default_python_submission_method(self) -> str:
|
||||
raise NotImplementedError("default_python_submission_method is not specified")
|
||||
|
||||
@available.parse_none
|
||||
@log_code_execution
|
||||
def submit_python_job(self, parsed_model: dict, compiled_code: str) -> AdapterResponse:
|
||||
submission_method = parsed_model["config"].get(
|
||||
|
||||
@@ -384,7 +384,7 @@ class RelationsCache:
|
||||
relation = self.relations.pop(old_key)
|
||||
new_key = new_relation.key()
|
||||
|
||||
# relaton has to rename its innards, so it needs the _CachedRelation.
|
||||
# relation has to rename its innards, so it needs the _CachedRelation.
|
||||
relation.rename(new_relation)
|
||||
# update all the relations that refer to it
|
||||
for cached in self.relations.values():
|
||||
|
||||
@@ -1,23 +1,17 @@
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from contextlib import contextmanager
|
||||
from importlib import import_module
|
||||
from typing import Type, Dict, Any, List, Optional, Set
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional, Set, Type
|
||||
|
||||
from dbt.exceptions import RuntimeException, InternalException
|
||||
from dbt.include.global_project import (
|
||||
PACKAGE_PATH as GLOBAL_PROJECT_PATH,
|
||||
PROJECT_NAME as GLOBAL_PROJECT_NAME,
|
||||
)
|
||||
from dbt.adapters.base.plugin import AdapterPlugin
|
||||
from dbt.adapters.protocol import AdapterConfig, AdapterProtocol, RelationProtocol
|
||||
from dbt.contracts.connection import AdapterRequiredConfig, Credentials
|
||||
from dbt.events.functions import fire_event
|
||||
from dbt.events.types import AdapterImportError, PluginLoadError
|
||||
from dbt.contracts.connection import Credentials, AdapterRequiredConfig
|
||||
from dbt.adapters.protocol import (
|
||||
AdapterProtocol,
|
||||
AdapterConfig,
|
||||
RelationProtocol,
|
||||
)
|
||||
from dbt.adapters.base.plugin import AdapterPlugin
|
||||
|
||||
from dbt.exceptions import InternalException, RuntimeException
|
||||
from dbt.include.global_project import PACKAGE_PATH as GLOBAL_PROJECT_PATH
|
||||
from dbt.include.global_project import PROJECT_NAME as GLOBAL_PROJECT_NAME
|
||||
|
||||
Adapter = AdapterProtocol
|
||||
|
||||
@@ -64,7 +58,7 @@ class AdapterContainer:
|
||||
# if we failed to import the target module in particular, inform
|
||||
# the user about it via a runtime error
|
||||
if exc.name == "dbt.adapters." + name:
|
||||
fire_event(AdapterImportError(exc=exc))
|
||||
fire_event(AdapterImportError(exc=str(exc)))
|
||||
raise RuntimeException(f"Could not find adapter type {name}!")
|
||||
# otherwise, the error had to have come from some underlying
|
||||
# library. Log the stack trace.
|
||||
@@ -217,3 +211,12 @@ def get_adapter_package_names(name: Optional[str]) -> List[str]:
|
||||
|
||||
def get_adapter_type_names(name: Optional[str]) -> List[str]:
|
||||
return FACTORY.get_adapter_type_names(name)
|
||||
|
||||
|
||||
@contextmanager
|
||||
def adapter_management():
|
||||
reset_adapters()
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
cleanup_connections()
|
||||
|
||||
@@ -88,7 +88,7 @@ class AdapterProtocol( # type: ignore[misc]
|
||||
],
|
||||
):
|
||||
# N.B. Technically these are ClassVars, but mypy doesn't support putting type vars in a
|
||||
# ClassVar due to the restirctiveness of PEP-526
|
||||
# ClassVar due to the restrictiveness of PEP-526
|
||||
# See: https://github.com/python/mypy/issues/5144
|
||||
AdapterSpecificConfigs: Type[AdapterConfig_T]
|
||||
Column: Type[Column_T]
|
||||
|
||||
44
core/dbt/cli/flags.py
Normal file
44
core/dbt/cli/flags.py
Normal file
@@ -0,0 +1,44 @@
|
||||
# TODO Move this to /core/dbt/flags.py when we're ready to break things
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from multiprocessing import get_context
|
||||
from pprint import pformat as pf
|
||||
|
||||
from click import get_current_context
|
||||
|
||||
if os.name != "nt":
|
||||
# https://bugs.python.org/issue41567
|
||||
import multiprocessing.popen_spawn_posix # type: ignore # noqa: F401
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Flags:
|
||||
def __init__(self, ctx=None) -> None:
|
||||
|
||||
if ctx is None:
|
||||
ctx = get_current_context()
|
||||
|
||||
def assign_params(ctx):
|
||||
"""Recursively adds all click params to flag object"""
|
||||
for param_name, param_value in ctx.params.items():
|
||||
# N.B. You have to use the base MRO method (object.__setattr__) to set attributes
|
||||
# when using frozen dataclasses.
|
||||
# https://docs.python.org/3/library/dataclasses.html#frozen-instances
|
||||
if hasattr(self, param_name):
|
||||
raise Exception(f"Duplicate flag names found in click command: {param_name}")
|
||||
object.__setattr__(self, param_name.upper(), param_value)
|
||||
if ctx.parent:
|
||||
assign_params(ctx.parent)
|
||||
|
||||
assign_params(ctx)
|
||||
|
||||
# Hard coded flags
|
||||
object.__setattr__(self, "WHICH", ctx.info_name)
|
||||
object.__setattr__(self, "MP_CONTEXT", get_context("spawn"))
|
||||
|
||||
# Support console DO NOT TRACK initiave
|
||||
if os.getenv("DO_NOT_TRACK", "").lower() in (1, "t", "true", "y", "yes"):
|
||||
object.__setattr__(self, "ANONYMOUS_USAGE_STATS", False)
|
||||
|
||||
def __str__(self) -> str:
|
||||
return str(pf(self.__dict__))
|
||||
@@ -1,44 +1,70 @@
|
||||
import click
|
||||
from dbt.cli import params as p
|
||||
import sys
|
||||
import inspect # This is temporary for RAT-ing
|
||||
from copy import copy
|
||||
from pprint import pformat as pf # This is temporary for RAT-ing
|
||||
|
||||
# This is temporary for RAT-ing
|
||||
import inspect
|
||||
from pprint import pformat as pf
|
||||
import click
|
||||
from dbt.adapters.factory import adapter_management
|
||||
from dbt.cli import params as p
|
||||
from dbt.cli.flags import Flags
|
||||
from dbt.profiler import profiler
|
||||
|
||||
|
||||
def cli_runner():
|
||||
# Alias "list" to "ls"
|
||||
ls = copy(cli.commands["list"])
|
||||
ls.hidden = True
|
||||
cli.add_command(ls, "ls")
|
||||
|
||||
# Run the cli
|
||||
cli()
|
||||
|
||||
|
||||
# dbt
|
||||
@click.group(
|
||||
context_settings={"help_option_names": ["-h", "--help"]},
|
||||
invoke_without_command=True,
|
||||
no_args_is_help=True,
|
||||
epilog="Specify one of these sub-commands and you can find more help from there.",
|
||||
)
|
||||
@click.pass_context
|
||||
@p.version
|
||||
@p.anonymous_usage_stats
|
||||
@p.cache_selected_only
|
||||
@p.debug
|
||||
@p.enable_legacy_logger
|
||||
@p.event_buffer_size
|
||||
@p.fail_fast
|
||||
@p.log_cache_events
|
||||
@p.log_format
|
||||
@p.macro_debugging
|
||||
@p.partial_parse
|
||||
@p.print
|
||||
@p.printer_width
|
||||
@p.quiet
|
||||
@p.send_anonymous_usage_stats
|
||||
@p.record_timing_info
|
||||
@p.static_parser
|
||||
@p.use_colors
|
||||
@p.use_experimental_parser
|
||||
@p.version
|
||||
@p.version_check
|
||||
@p.warn_error
|
||||
@p.write_json
|
||||
@p.event_buffer_size
|
||||
@p.record_timing
|
||||
def cli(ctx, **kwargs):
|
||||
"""An ELT tool for managing your SQL transformations and data models.
|
||||
For more documentation on these commands, visit: docs.getdbt.com
|
||||
"""
|
||||
if kwargs.get("version", False):
|
||||
incomplete_flags = Flags()
|
||||
|
||||
# Profiling
|
||||
if incomplete_flags.RECORD_TIMING_INFO:
|
||||
ctx.with_resource(profiler(enable=True, outfile=incomplete_flags.RECORD_TIMING_INFO))
|
||||
|
||||
# Adapter management
|
||||
ctx.with_resource(adapter_management())
|
||||
|
||||
# Version info
|
||||
if incomplete_flags.VERSION:
|
||||
click.echo(f"`version` called\n ctx.params: {pf(ctx.params)}")
|
||||
sys.exit()
|
||||
return
|
||||
else:
|
||||
del ctx.params["version"]
|
||||
|
||||
@@ -46,26 +72,43 @@ def cli(ctx, **kwargs):
|
||||
# dbt build
|
||||
@cli.command("build")
|
||||
@click.pass_context
|
||||
@p.defer
|
||||
@p.exclude
|
||||
@p.fail_fast
|
||||
@p.full_refresh
|
||||
@p.indirect_selection
|
||||
@p.log_path
|
||||
@p.models
|
||||
@p.profile
|
||||
@p.profiles_dir
|
||||
@p.project_dir
|
||||
@p.selector
|
||||
@p.show
|
||||
@p.state
|
||||
@p.store_failures
|
||||
@p.target
|
||||
@p.target_path
|
||||
@p.threads
|
||||
@p.vars
|
||||
@p.version_check
|
||||
def build(ctx, **kwargs):
|
||||
"""Run all Seeds, Models, Snapshots, and tests in DAG order"""
|
||||
click.echo(
|
||||
f"`{inspect.stack()[0][3]}` called\n kwargs: {kwargs}\n ctx: {pf(ctx.parent.params)}"
|
||||
)
|
||||
flags = Flags()
|
||||
click.echo(f"`{inspect.stack()[0][3]}` called\n flags: {flags}")
|
||||
|
||||
|
||||
# dbt clean
|
||||
@cli.command("clean")
|
||||
@click.pass_context
|
||||
@p.project_dir
|
||||
@p.profiles_dir
|
||||
@p.profile
|
||||
@p.profiles_dir
|
||||
@p.project_dir
|
||||
@p.target
|
||||
@p.vars
|
||||
def clean(ctx, **kwargs):
|
||||
"""Delete all folders in the clean-targets list (usually the dbt_packages and target directories.)"""
|
||||
click.echo(
|
||||
f"`{inspect.stack()[0][3]}` called\n kwargs: {kwargs}\n ctx: {pf(ctx.parent.params)}"
|
||||
)
|
||||
flags = Flags()
|
||||
click.echo(f"`{inspect.stack()[0][3]}` called\n flags: {flags}")
|
||||
|
||||
|
||||
# dbt docs
|
||||
@@ -78,86 +121,82 @@ def docs(ctx, **kwargs):
|
||||
# dbt docs generate
|
||||
@docs.command("generate")
|
||||
@click.pass_context
|
||||
@p.version_check
|
||||
@p.project_dir
|
||||
@p.profiles_dir
|
||||
@p.profile
|
||||
@p.target
|
||||
@p.vars
|
||||
@p.compile_docs
|
||||
@p.defer
|
||||
@p.threads
|
||||
@p.target_path
|
||||
@p.exclude
|
||||
@p.log_path
|
||||
@p.models
|
||||
@p.exclude
|
||||
@p.profile
|
||||
@p.profiles_dir
|
||||
@p.project_dir
|
||||
@p.selector
|
||||
@p.state
|
||||
@p.target
|
||||
@p.target_path
|
||||
@p.threads
|
||||
@p.vars
|
||||
@p.version_check
|
||||
def docs_generate(ctx, **kwargs):
|
||||
"""Generate the documentation website for your project"""
|
||||
click.echo(
|
||||
f"`{inspect.stack()[0][3]}` called\n kwargs: {kwargs}\n ctx: {pf(ctx.parent.parent.params)}"
|
||||
)
|
||||
flags = Flags()
|
||||
click.echo(f"`{inspect.stack()[0][3]}` called\n flags: {flags}")
|
||||
|
||||
|
||||
# dbt docs serve
|
||||
@docs.command("serve")
|
||||
@click.pass_context
|
||||
@p.project_dir
|
||||
@p.profiles_dir
|
||||
@p.browser
|
||||
@p.port
|
||||
@p.profile
|
||||
@p.profiles_dir
|
||||
@p.project_dir
|
||||
@p.target
|
||||
@p.vars
|
||||
@p.port
|
||||
@p.browser
|
||||
def docs_serve(ctx, **kwargs):
|
||||
"""Serve the documentation website for your project"""
|
||||
click.echo(
|
||||
f"`{inspect.stack()[0][3]}` called\n kwargs: {kwargs}\n ctx: {pf(ctx.parent.parent.params)}"
|
||||
)
|
||||
flags = Flags()
|
||||
click.echo(f"`{inspect.stack()[0][3]}` called\n flags: {flags}")
|
||||
|
||||
|
||||
# dbt compile
|
||||
@cli.command("compile")
|
||||
@click.pass_context
|
||||
@p.version_check
|
||||
@p.project_dir
|
||||
@p.profiles_dir
|
||||
@p.profile
|
||||
@p.target
|
||||
@p.vars
|
||||
@p.parse_only
|
||||
@p.threads
|
||||
@p.target_path
|
||||
@p.defer
|
||||
@p.exclude
|
||||
@p.full_refresh
|
||||
@p.log_path
|
||||
@p.models
|
||||
@p.exclude
|
||||
@p.parse_only
|
||||
@p.profile
|
||||
@p.profiles_dir
|
||||
@p.project_dir
|
||||
@p.selector
|
||||
@p.state
|
||||
@p.defer
|
||||
@p.full_refresh
|
||||
@p.target
|
||||
@p.target_path
|
||||
@p.threads
|
||||
@p.vars
|
||||
@p.version_check
|
||||
def compile(ctx, **kwargs):
|
||||
"""Generates executable SQL from source, model, test, and analysis files. Compiled SQL files are written to the target/ directory."""
|
||||
click.echo(
|
||||
f"`{inspect.stack()[0][3]}` called\n kwargs: {kwargs}\n ctx: {pf(ctx.parent.params)}"
|
||||
)
|
||||
flags = Flags()
|
||||
click.echo(f"`{inspect.stack()[0][3]}` called\n flags: {flags}")
|
||||
|
||||
|
||||
# dbt debug
|
||||
@cli.command("debug")
|
||||
@click.pass_context
|
||||
@p.version_check
|
||||
@p.project_dir
|
||||
@p.profiles_dir
|
||||
@p.config_dir
|
||||
@p.profile
|
||||
@p.profiles_dir
|
||||
@p.project_dir
|
||||
@p.target
|
||||
@p.vars
|
||||
@p.config_dir
|
||||
@p.version_check
|
||||
def debug(ctx, **kwargs):
|
||||
"""Show some helpful information about dbt for debugging. Not to be confused with the --debug option which increases verbosity."""
|
||||
click.echo(
|
||||
f"`{inspect.stack()[0][3]}` called\n kwargs: {kwargs}\n ctx: {pf(ctx.parent.params)}"
|
||||
)
|
||||
flags = Flags()
|
||||
click.echo(f"`{inspect.stack()[0][3]}` called\n flags: {flags}")
|
||||
|
||||
|
||||
# dbt deps
|
||||
@@ -170,9 +209,8 @@ def debug(ctx, **kwargs):
|
||||
@p.vars
|
||||
def deps(ctx, **kwargs):
|
||||
"""Pull the most recent version of the dependencies listed in packages.yml"""
|
||||
click.echo(
|
||||
f"`{inspect.stack()[0][3]}` called\n kwargs: {kwargs}\n ctx: {pf(ctx.parent.params)}"
|
||||
)
|
||||
flags = Flags()
|
||||
click.echo(f"`{inspect.stack()[0][3]}` called\n flags: {flags}")
|
||||
|
||||
|
||||
# dbt init
|
||||
@@ -181,147 +219,139 @@ def deps(ctx, **kwargs):
|
||||
@p.profile
|
||||
@p.profiles_dir
|
||||
@p.project_dir
|
||||
@p.skip_profile_setup
|
||||
@p.target
|
||||
@p.vars
|
||||
@p.skip_profile_setup
|
||||
def init(ctx, **kwargs):
|
||||
"""Initialize a new DBT project."""
|
||||
click.echo(
|
||||
f"`{inspect.stack()[0][3]}` called\n kwargs: {kwargs}\n ctx: {pf(ctx.parent.params)}"
|
||||
)
|
||||
flags = Flags()
|
||||
click.echo(f"`{inspect.stack()[0][3]}` called\n flags: {flags}")
|
||||
|
||||
|
||||
# dbt list
|
||||
# dbt TODO: Figure out aliasing for ls (or just c/p?)
|
||||
@cli.command("list")
|
||||
@click.pass_context
|
||||
@p.exclude
|
||||
@p.indirect_selection
|
||||
@p.models
|
||||
@p.output
|
||||
@p.output_keys
|
||||
@p.profile
|
||||
@p.profiles_dir
|
||||
@p.project_dir
|
||||
@p.target
|
||||
@p.vars
|
||||
@p.output
|
||||
@p.ouptut_keys
|
||||
@p.resource_type
|
||||
@p.models
|
||||
@p.indirect_selection
|
||||
@p.exclude
|
||||
@p.selector
|
||||
@p.state
|
||||
@p.target
|
||||
@p.vars
|
||||
def list(ctx, **kwargs):
|
||||
"""List the resources in your project"""
|
||||
click.echo(
|
||||
f"`{inspect.stack()[0][3]}` called\n kwargs: {kwargs}\n ctx: {pf(ctx.parent.params)}"
|
||||
)
|
||||
flags = Flags()
|
||||
click.echo(f"`{inspect.stack()[0][3]}` called\n flags: {flags}")
|
||||
|
||||
|
||||
# dbt parse
|
||||
@cli.command("parse")
|
||||
@click.pass_context
|
||||
@p.compile_parse
|
||||
@p.log_path
|
||||
@p.profile
|
||||
@p.profiles_dir
|
||||
@p.project_dir
|
||||
@p.target
|
||||
@p.vars
|
||||
@p.write_manifest
|
||||
@p.compile_parse
|
||||
@p.threads
|
||||
@p.target_path
|
||||
@p.log_path
|
||||
@p.threads
|
||||
@p.vars
|
||||
@p.version_check
|
||||
@p.write_manifest
|
||||
def parse(ctx, **kwargs):
|
||||
"""Parses the project and provides information on performance"""
|
||||
click.echo(
|
||||
f"`{inspect.stack()[0][3]}` called\n kwargs: {kwargs}\n ctx: {pf(ctx.parent.params)}"
|
||||
)
|
||||
flags = Flags()
|
||||
click.echo(f"`{inspect.stack()[0][3]}` called\n flags: {flags}")
|
||||
|
||||
|
||||
# dbt run
|
||||
@cli.command("run")
|
||||
@click.pass_context
|
||||
@p.defer
|
||||
@p.exclude
|
||||
@p.fail_fast
|
||||
@p.version_check
|
||||
@p.full_refresh
|
||||
@p.log_path
|
||||
@p.models
|
||||
@p.profile
|
||||
@p.profiles_dir
|
||||
@p.project_dir
|
||||
@p.target
|
||||
@p.vars
|
||||
@p.log_path
|
||||
@p.target_path
|
||||
@p.threads
|
||||
@p.models
|
||||
@p.exclude
|
||||
@p.selector
|
||||
@p.state
|
||||
@p.defer
|
||||
@p.full_refresh
|
||||
@p.target
|
||||
@p.target_path
|
||||
@p.threads
|
||||
@p.vars
|
||||
@p.version_check
|
||||
def run(ctx, **kwargs):
|
||||
"""Compile SQL and execute against the current target database."""
|
||||
click.echo(
|
||||
f"`{inspect.stack()[0][3]}` called\n kwargs: {kwargs}\n ctx: {pf(ctx.parent.params)}"
|
||||
)
|
||||
flags = Flags()
|
||||
click.echo(f"`{inspect.stack()[0][3]}` called\n flags: {flags}")
|
||||
|
||||
|
||||
# dbt run operation
|
||||
@cli.command("run-operation")
|
||||
@click.pass_context
|
||||
@p.args
|
||||
@p.profile
|
||||
@p.profiles_dir
|
||||
@p.project_dir
|
||||
@p.target
|
||||
@p.vars
|
||||
@p.args
|
||||
def run_operation(ctx, **kwargs):
|
||||
"""Run the named macro with any supplied arguments."""
|
||||
click.echo(
|
||||
f"`{inspect.stack()[0][3]}` called\n kwargs: {kwargs}\n ctx: {pf(ctx.parent.params)}"
|
||||
)
|
||||
flags = Flags()
|
||||
click.echo(f"`{inspect.stack()[0][3]}` called\n flags: {flags}")
|
||||
|
||||
|
||||
# dbt seed
|
||||
@cli.command("seed")
|
||||
@click.pass_context
|
||||
@p.version_check
|
||||
@p.exclude
|
||||
@p.full_refresh
|
||||
@p.log_path
|
||||
@p.models
|
||||
@p.profile
|
||||
@p.profiles_dir
|
||||
@p.project_dir
|
||||
@p.selector
|
||||
@p.show
|
||||
@p.state
|
||||
@p.target
|
||||
@p.vars
|
||||
@p.full_refresh
|
||||
@p.log_path
|
||||
@p.target_path
|
||||
@p.threads
|
||||
@p.models
|
||||
@p.exclude
|
||||
@p.selector
|
||||
@p.state
|
||||
@p.show
|
||||
@p.vars
|
||||
@p.version_check
|
||||
def seed(ctx, **kwargs):
|
||||
"""Load data from csv files into your data warehouse."""
|
||||
click.echo(
|
||||
f"`{inspect.stack()[0][3]}` called\n kwargs: {kwargs}\n ctx: {pf(ctx.parent.params)}"
|
||||
)
|
||||
flags = Flags()
|
||||
click.echo(f"`{inspect.stack()[0][3]}` called\n flags: {flags}")
|
||||
|
||||
|
||||
# dbt snapshot
|
||||
@cli.command("snapshot")
|
||||
@click.pass_context
|
||||
@p.defer
|
||||
@p.exclude
|
||||
@p.models
|
||||
@p.profile
|
||||
@p.profiles_dir
|
||||
@p.project_dir
|
||||
@p.target
|
||||
@p.vars
|
||||
@p.threads
|
||||
@p.models
|
||||
@p.exclude
|
||||
@p.selector
|
||||
@p.state
|
||||
@p.defer
|
||||
@p.target
|
||||
@p.threads
|
||||
@p.vars
|
||||
def snapshot(ctx, **kwargs):
|
||||
"""Execute snapshots defined in your project"""
|
||||
click.echo(
|
||||
f"`{inspect.stack()[0][3]}` called\n kwargs: {kwargs}\n ctx: {pf(ctx.parent.params)}"
|
||||
)
|
||||
flags = Flags()
|
||||
click.echo(f"`{inspect.stack()[0][3]}` called\n flags: {flags}")
|
||||
|
||||
|
||||
# dbt source
|
||||
@@ -334,51 +364,49 @@ def source(ctx, **kwargs):
|
||||
# dbt source freshness
|
||||
@source.command("freshness")
|
||||
@click.pass_context
|
||||
@p.exclude
|
||||
@p.models
|
||||
@p.output_path # TODO: Is this ok to re-use? We have three different output params, how much can we consolidate?
|
||||
@p.profile
|
||||
@p.profiles_dir
|
||||
@p.project_dir
|
||||
@p.target
|
||||
@p.vars
|
||||
@p.threads
|
||||
@p.models
|
||||
@p.exclude
|
||||
@p.selector
|
||||
@p.state
|
||||
@p.output_path # TODO: Is this ok to re-use? We have three different output params, how much can we consolidate?
|
||||
@p.target
|
||||
@p.threads
|
||||
@p.vars
|
||||
def freshness(ctx, **kwargs):
|
||||
"""Snapshots the current freshness of the project's sources"""
|
||||
click.echo(
|
||||
f"`{inspect.stack()[0][3]}` called\n kwargs: {kwargs}\n ctx: {pf(ctx.parent.parent.params)}"
|
||||
)
|
||||
flags = Flags()
|
||||
click.echo(f"`{inspect.stack()[0][3]}` called\n flags: {flags}")
|
||||
|
||||
|
||||
# dbt test
|
||||
@cli.command("test")
|
||||
@click.pass_context
|
||||
@p.defer
|
||||
@p.exclude
|
||||
@p.fail_fast
|
||||
@p.version_check
|
||||
@p.store_failures
|
||||
@p.indirect_selection
|
||||
@p.log_path
|
||||
@p.models
|
||||
@p.profile
|
||||
@p.profiles_dir
|
||||
@p.project_dir
|
||||
@p.target
|
||||
@p.vars
|
||||
@p.indirect_selection
|
||||
@p.log_path
|
||||
@p.target_path
|
||||
@p.threads
|
||||
@p.models
|
||||
@p.exclude
|
||||
@p.selector
|
||||
@p.state
|
||||
@p.defer
|
||||
@p.store_failures
|
||||
@p.target
|
||||
@p.target_path
|
||||
@p.threads
|
||||
@p.vars
|
||||
@p.version_check
|
||||
def test(ctx, **kwargs):
|
||||
"""Runs tests on data in deployed models. Run this after `dbt run`"""
|
||||
click.echo(
|
||||
f"`{inspect.stack()[0][3]}` called\n kwargs: {kwargs}\n ctx: {pf(ctx.parent.params)}"
|
||||
)
|
||||
flags = Flags()
|
||||
click.echo(f"`{inspect.stack()[0][3]}` called\n flags: {flags}")
|
||||
|
||||
|
||||
# Support running as a module
|
||||
if __name__ == "__main__":
|
||||
cli()
|
||||
cli_runner()
|
||||
|
||||
33
core/dbt/cli/option_types.py
Normal file
33
core/dbt/cli/option_types.py
Normal file
@@ -0,0 +1,33 @@
|
||||
from click import ParamType
|
||||
import yaml
|
||||
|
||||
|
||||
class YAML(ParamType):
|
||||
"""The Click YAML type. Converts YAML strings into objects."""
|
||||
|
||||
name = "YAML"
|
||||
|
||||
def convert(self, value, param, ctx):
|
||||
# assume non-string values are a problem
|
||||
if not isinstance(value, str):
|
||||
self.fail(f"Cannot load YAML from type {type(value)}", param, ctx)
|
||||
try:
|
||||
return yaml.load(value, Loader=yaml.Loader)
|
||||
except yaml.parser.ParserError:
|
||||
self.fail(f"String '{value}' is not valid YAML", param, ctx)
|
||||
|
||||
|
||||
class Truthy(ParamType):
|
||||
"""The Click Truthy type. Converts strings into a "truthy" type"""
|
||||
|
||||
name = "TRUTHY"
|
||||
|
||||
def convert(self, value, param, ctx):
|
||||
# assume non-string / non-None values are a problem
|
||||
if not isinstance(value, (str, None)):
|
||||
self.fail(f"Cannot load TRUTHY from type {type(value)}", param, ctx)
|
||||
|
||||
if value is None or value.lower() in ("0", "false", "f"):
|
||||
return None
|
||||
else:
|
||||
return value
|
||||
@@ -1,56 +1,58 @@
|
||||
import click
|
||||
import yaml
|
||||
from pathlib import Path, PurePath
|
||||
from click import ParamType
|
||||
|
||||
import click
|
||||
from dbt.cli.option_types import YAML
|
||||
from dbt.cli.resolvers import default_project_dir, default_profiles_dir
|
||||
|
||||
|
||||
class YAML(ParamType):
|
||||
"""The Click YAML type. Converts YAML strings into objects."""
|
||||
|
||||
name = "YAML"
|
||||
|
||||
def convert(self, value, param, ctx):
|
||||
# assume non-string values are a problem
|
||||
if not isinstance(value, str):
|
||||
self.fail(f"Cannot load YAML from type {type(value)}", param, ctx)
|
||||
try:
|
||||
return yaml.load(value, Loader=yaml.Loader)
|
||||
except yaml.parser.ParserError:
|
||||
self.fail(f"String '{value}' is not valid YAML", param, ctx)
|
||||
|
||||
# TODO: The name (reflected in flags) is a correction!
|
||||
# The original name was `SEND_ANONYMOUS_USAGE_STATS` and used an env var called "DBT_SEND_ANONYMOUS_USAGE_STATS"
|
||||
# Both of which break existing naming conventions (doesn't match param flag).
|
||||
# This will need to be fixed before use in the main codebase and communicated as a change to the community!
|
||||
anonymous_usage_stats = click.option(
|
||||
"--anonymous-usage-stats/--no-anonymous-usage-stats",
|
||||
envvar="DBT_ANONYMOUS_USAGE_STATS",
|
||||
help="Send anonymous usage stats to dbt Labs.",
|
||||
default=True,
|
||||
)
|
||||
|
||||
args = click.option(
|
||||
"--args",
|
||||
envvar=None,
|
||||
help="Supply arguments to the macro. This dictionary will be mapped to the keyword arguments defined in the selected macro. This argument should be a YAML string, eg. '{my_variable: my_value}'",
|
||||
type=YAML(),
|
||||
)
|
||||
|
||||
browser = click.option(
|
||||
"--browser/--no-browser",
|
||||
envvar=None,
|
||||
help="Wether or not to open a local web browser after starting the server",
|
||||
default=True,
|
||||
)
|
||||
|
||||
cache_selected_only = click.option(
|
||||
"--cache-selected-only/--no-cache-selected-only",
|
||||
envvar="DBT_CACHE_SELECTED_ONLY",
|
||||
help="Pre cache database objects relevant to selected resource only.",
|
||||
default=False,
|
||||
)
|
||||
|
||||
compile_docs = click.option(
|
||||
"--compile/--no-compile",
|
||||
envvar=None,
|
||||
help="Wether or not to run 'dbt compile' as part of docs generation",
|
||||
default=True,
|
||||
)
|
||||
|
||||
compile_parse = click.option(
|
||||
"--compile/--no-compile",
|
||||
envvar=None,
|
||||
help="TODO: No help text currently available",
|
||||
default=True,
|
||||
)
|
||||
|
||||
config_dir = click.option(
|
||||
"--config-dir",
|
||||
envvar=None,
|
||||
help="If specified, DBT will show path information for this project",
|
||||
type=click.STRING,
|
||||
)
|
||||
@@ -58,44 +60,68 @@ config_dir = click.option(
|
||||
debug = click.option(
|
||||
"--debug/--no-debug",
|
||||
"-d/ ",
|
||||
envvar="DBT_DEBUG",
|
||||
help="Display debug logging during dbt execution. Useful for debugging and making bug reports.",
|
||||
default=False,
|
||||
)
|
||||
|
||||
# TODO: The env var and name (reflected in flags) are corrections!
|
||||
# The original name was `DEFER_MODE` and used an env var called "DBT_DEFER_TO_STATE"
|
||||
# Both of which break existing naming conventions.
|
||||
# This will need to be fixed before use in the main codebase and communicated as a change to the community!
|
||||
defer = click.option(
|
||||
"--defer/--no-defer",
|
||||
envvar="DBT_DEFER",
|
||||
help="If set, defer to the state variable for resolving unselected nodes.",
|
||||
default=True,
|
||||
)
|
||||
|
||||
enable_legacy_logger = click.option(
|
||||
"--enable-legacy-logger/--no-enable-legacy-logger",
|
||||
envvar="DBT_ENABLE_LEGACY_LOGGER",
|
||||
hidden=True,
|
||||
)
|
||||
|
||||
event_buffer_size = click.option(
|
||||
"--event-buffer-size",
|
||||
envvar="DBT_EVENT_BUFFER_SIZE",
|
||||
help="Sets the max number of events to buffer in EVENT_HISTORY.",
|
||||
default=100000,
|
||||
type=click.INT,
|
||||
)
|
||||
|
||||
exclude = click.option("--exclude", help="Specify the nodes to exclude.")
|
||||
exclude = click.option("--exclude", envvar=None, help="Specify the nodes to exclude.")
|
||||
|
||||
fail_fast = click.option(
|
||||
"--fail-fast/--no-fail-fast", "-x/ ", help="Stop execution on first failure.", default=False
|
||||
"--fail-fast/--no-fail-fast",
|
||||
"-x/ ",
|
||||
envvar="DBT_FAIL_FAST",
|
||||
help="Stop execution on first failure.",
|
||||
)
|
||||
|
||||
full_refresh = click.option(
|
||||
"--full-refresh",
|
||||
"-f",
|
||||
envvar="DBT_FULL_REFRESH",
|
||||
help="If specified, dbt will drop incremental models and fully-recalculate the incremental table from the model definition.",
|
||||
is_flag=True,
|
||||
)
|
||||
|
||||
indirect_selection = click.option(
|
||||
"--indirect_selection",
|
||||
"--indirect-selection",
|
||||
envvar="DBT_INDIRECT_SELECTION",
|
||||
help="Select all tests that are adjacent to selected resources, even if they those resources have been explicitly selected.",
|
||||
type=click.Choice(["eager", "cautious"], case_sensitive=False),
|
||||
default="eager",
|
||||
)
|
||||
|
||||
log_cache_events = click.option(
|
||||
"--log-cache-events/--no-log-cache-events",
|
||||
help="Enable verbose adapter cache logging.",
|
||||
envvar="DBT_LOG_CACHE_EVENTS",
|
||||
)
|
||||
|
||||
log_format = click.option(
|
||||
"--log-format",
|
||||
envvar="DBT_LOG_FORMAT",
|
||||
help="Specify the log format, overriding the command's default.",
|
||||
type=click.Choice(["text", "json", "default"], case_sensitive=False),
|
||||
default="default",
|
||||
@@ -103,28 +129,42 @@ log_format = click.option(
|
||||
|
||||
log_path = click.option(
|
||||
"--log-path",
|
||||
envvar="DBT_LOG_PATH",
|
||||
help="Configure the 'log-path'. Only applies this setting for the current run. Overrides the 'DBT_LOG_PATH' if it is set.",
|
||||
type=click.Path(),
|
||||
)
|
||||
|
||||
models = click.option("-m", "-s", help="Specify the nodes to include.", multiple=True)
|
||||
macro_debugging = click.option(
|
||||
"--macro-debugging/--no-macro-debugging",
|
||||
envvar="DBT_MACRO_DEBUGGING",
|
||||
hidden=True,
|
||||
)
|
||||
|
||||
models = click.option(
|
||||
"-m",
|
||||
"-s",
|
||||
"models",
|
||||
envvar=None,
|
||||
help="Specify the nodes to include.",
|
||||
multiple=True,
|
||||
)
|
||||
|
||||
output = click.option(
|
||||
"--output",
|
||||
envvar=None,
|
||||
help="TODO: No current help text",
|
||||
type=click.Choice(["json", "name", "path", "selector"], case_sensitive=False),
|
||||
default="name",
|
||||
)
|
||||
|
||||
ouptut_keys = click.option(
|
||||
"--output-keys",
|
||||
help="TODO: No current help text",
|
||||
default=False,
|
||||
output_keys = click.option(
|
||||
"--output-keys", envvar=None, help="TODO: No current help text", type=click.STRING
|
||||
)
|
||||
|
||||
output_path = click.option(
|
||||
"--output",
|
||||
"-o",
|
||||
envvar=None,
|
||||
help="Specify the output path for the json report. By default, outputs to 'target/sources.json'",
|
||||
type=click.Path(file_okay=True, dir_okay=False, writable=True),
|
||||
default=PurePath.joinpath(Path.cwd(), "target/sources.json"),
|
||||
@@ -132,65 +172,84 @@ output_path = click.option(
|
||||
|
||||
parse_only = click.option(
|
||||
"--parse-only",
|
||||
envvar=None,
|
||||
help="TODO: No help text currently available",
|
||||
is_flag=True,
|
||||
)
|
||||
|
||||
partial_parse = click.option(
|
||||
"--partial-parse/--no-partial-parse",
|
||||
envvar="DBT_PARTIAL_PARSE",
|
||||
help="Allow for partial parsing by looking for and writing to a pickle file in the target directory. This overrides the user configuration file.",
|
||||
default=True,
|
||||
)
|
||||
|
||||
port = click.option(
|
||||
"--port", help="Specify the port number for the docs server", default=8080, type=click.INT
|
||||
"--port",
|
||||
envvar=None,
|
||||
help="Specify the port number for the docs server",
|
||||
default=8080,
|
||||
type=click.INT,
|
||||
)
|
||||
|
||||
# TODO: The env var and name (reflected in flags) are corrections!
|
||||
# The original name was `NO_PRINT` and used the env var `DBT_NO_PRINT`.
|
||||
# Both of which break existing naming conventions.
|
||||
# This will need to be fixed before use in the main codebase and communicated as a change to the community!
|
||||
print = click.option(
|
||||
"--print/--no-print", help="Output all {{ print() }} macro calls.", default=True
|
||||
"--print/--no-print",
|
||||
envvar="DBT_PRINT",
|
||||
help="Output all {{ print() }} macro calls.",
|
||||
default=True,
|
||||
)
|
||||
|
||||
printer_width = click.option(
|
||||
"--printer_width", help="Sets the width of terminal output", type=click.INT, default=80
|
||||
"--printer-width",
|
||||
envvar="DBT_PRINTER_WIDTH",
|
||||
help="Sets the width of terminal output",
|
||||
type=click.INT,
|
||||
default=80,
|
||||
)
|
||||
|
||||
profile = click.option(
|
||||
"--profile",
|
||||
envvar=None,
|
||||
help="Which profile to load. Overrides setting in dbt_project.yml.",
|
||||
)
|
||||
|
||||
profiles_dir = click.option(
|
||||
"--profiles-dir",
|
||||
help=f"Which directory to look in for the profiles.yml file. Default = {PurePath.joinpath(Path.home(), '.dbt')}",
|
||||
default=PurePath.joinpath(Path.home(), ".dbt"),
|
||||
type=click.Path(
|
||||
exists=True,
|
||||
),
|
||||
envvar="DBT_PROFILES_DIR",
|
||||
help="Which directory to look in for the profiles.yml file. If not set, dbt will look in the current working directory first, then HOME/.dbt/",
|
||||
default=default_profiles_dir(),
|
||||
type=click.Path(exists=True),
|
||||
)
|
||||
|
||||
project_dir = click.option(
|
||||
"--project-dir",
|
||||
envvar=None,
|
||||
help="Which directory to look in for the dbt_project.yml file. Default is the current working directory and its parents.",
|
||||
default=Path.cwd(),
|
||||
default=default_project_dir(),
|
||||
type=click.Path(exists=True),
|
||||
)
|
||||
|
||||
quiet = click.option(
|
||||
"--quiet/--no-quiet",
|
||||
envvar="DBT_QUIET",
|
||||
help="Suppress all non-error logging to stdout. Does not affect {{ print() }} macro calls.",
|
||||
default=False,
|
||||
)
|
||||
|
||||
record_timing = click.option(
|
||||
"-r",
|
||||
record_timing_info = click.option(
|
||||
"--record-timing-info",
|
||||
"-r",
|
||||
envvar=None,
|
||||
help="When this option is passed, dbt will output low-level timing stats to the specified file. Example: `--record-timing-info output.profile`",
|
||||
is_flag=True,
|
||||
default=False,
|
||||
type=click.Path(exists=False),
|
||||
)
|
||||
|
||||
resource_type = click.option(
|
||||
"--resource-type",
|
||||
envvar=None,
|
||||
help="TODO: No current help text",
|
||||
type=click.Choice(
|
||||
[
|
||||
@@ -210,50 +269,63 @@ resource_type = click.option(
|
||||
default="default",
|
||||
)
|
||||
|
||||
selector = click.option("--selector", help="The selector name to use, as defined in selectors.yml")
|
||||
|
||||
send_anonymous_usage_stats = click.option(
|
||||
"--anonymous-usage-stats/--no-anonymous-usage-stats",
|
||||
help="Send anonymous usage stats to dbt Labs.",
|
||||
default=True,
|
||||
selector = click.option(
|
||||
"--selector", envvar=None, help="The selector name to use, as defined in selectors.yml"
|
||||
)
|
||||
|
||||
show = click.option(
|
||||
"--show",
|
||||
help="Show a sample of the loaded data in the terminal",
|
||||
default=False,
|
||||
"--show", envvar=None, help="Show a sample of the loaded data in the terminal", is_flag=True
|
||||
)
|
||||
|
||||
skip_profile_setup = click.option(
|
||||
"--skip-profile-setup",
|
||||
"-s",
|
||||
help="Skip interative profile setup.",
|
||||
default=False,
|
||||
"--skip-profile-setup", "-s", envvar=None, help="Skip interative profile setup.", is_flag=True
|
||||
)
|
||||
|
||||
# TODO: The env var and name (reflected in flags) are corrections!
|
||||
# The original name was `ARTIFACT_STATE_PATH` and used the env var `DBT_ARTIFACT_STATE_PATH`.
|
||||
# Both of which break existing naming conventions.
|
||||
# This will need to be fixed before use in the main codebase and communicated as a change to the community!
|
||||
state = click.option(
|
||||
"--state",
|
||||
envvar="DBT_STATE",
|
||||
help="If set, use the given directory as the source for json files to compare with this project.",
|
||||
type=click.Path(
|
||||
dir_okay=True,
|
||||
exists=True,
|
||||
file_okay=False,
|
||||
readable=True,
|
||||
resolve_path=True,
|
||||
),
|
||||
)
|
||||
|
||||
static_parser = click.option(
|
||||
"--static-parser/--no-static-parser", help="Use the static parser.", default=True
|
||||
"--static-parser/--no-static-parser",
|
||||
envvar="DBT_STATIC_PARSER",
|
||||
help="Use the static parser.",
|
||||
default=True,
|
||||
)
|
||||
|
||||
store_failures = click.option(
|
||||
"--store-failures", help="Store test results (failing rows) in the database", default=False
|
||||
"--store-failures",
|
||||
envvar="DBT_STORE_FAILURES",
|
||||
help="Store test results (failing rows) in the database",
|
||||
is_flag=True,
|
||||
)
|
||||
|
||||
target = click.option("-t", "--target", help="Which target to load for the given profile")
|
||||
target = click.option(
|
||||
"--target", "-t", envvar=None, help="Which target to load for the given profile"
|
||||
)
|
||||
|
||||
target_path = click.option(
|
||||
"--target-path",
|
||||
envvar="DBT_TARGET_PATH",
|
||||
help="Configure the 'target-path'. Only applies this setting for the current run. Overrides the 'DBT_TARGET_PATH' if it is set.",
|
||||
type=click.Path(),
|
||||
)
|
||||
|
||||
threads = click.option(
|
||||
"--threads",
|
||||
envvar=None,
|
||||
help="Specify number of threads to use while executing models. Overrides settings in profiles.yml.",
|
||||
default=1,
|
||||
type=click.INT,
|
||||
@@ -261,44 +333,54 @@ threads = click.option(
|
||||
|
||||
use_colors = click.option(
|
||||
"--use-colors/--no-use-colors",
|
||||
envvar="DBT_USE_COLORS",
|
||||
help="Output is colorized by default and may also be set in a profile or at the command line.",
|
||||
default=True,
|
||||
)
|
||||
|
||||
use_experimental_parser = click.option(
|
||||
"--use-experimental-parser/--no-use-experimental-parser",
|
||||
envvar="DBT_USE_EXPERIMENTAL_PARSER",
|
||||
help="Enable experimental parsing features.",
|
||||
default=False,
|
||||
)
|
||||
|
||||
vars = click.option(
|
||||
"--vars",
|
||||
envvar=None,
|
||||
help="Supply variables to the project. This argument overrides variables defined in your dbt_project.yml file. This argument should be a YAML string, eg. '{my_variable: my_value}'",
|
||||
type=YAML(),
|
||||
)
|
||||
|
||||
version = click.option("--version", help="Show version information", is_flag=True, default=False)
|
||||
version = click.option(
|
||||
"--version",
|
||||
envvar=None,
|
||||
help="Show version information",
|
||||
is_flag=True,
|
||||
)
|
||||
|
||||
version_check = click.option(
|
||||
"--version-check/--no-version-check",
|
||||
envvar="DBT_VERSION_CHECK",
|
||||
help="Ensure dbt's version matches the one specified in the dbt_project.yml file ('require-dbt-version')",
|
||||
default=True,
|
||||
)
|
||||
|
||||
warn_error = click.option(
|
||||
"--warn-error/--no-warn-error",
|
||||
envvar="DBT_WARN_ERROR",
|
||||
help="If dbt would normally warn, instead raise an exception. Examples include --models that selects nothing, deprecations, configurations with no associated models, invalid test configurations, and missing sources/refs in tests.",
|
||||
default=False,
|
||||
)
|
||||
|
||||
write_json = click.option(
|
||||
"--write-json/--no-write-json",
|
||||
envvar="DBT_WRITE_JSON",
|
||||
help="Writing the manifest and run_results.json files to disk",
|
||||
default=True,
|
||||
)
|
||||
|
||||
write_manifest = click.option(
|
||||
"--write-manifest/--no-write-manifest",
|
||||
envvar=None,
|
||||
help="TODO: No help text currently available",
|
||||
default=True,
|
||||
)
|
||||
|
||||
11
core/dbt/cli/resolvers.py
Normal file
11
core/dbt/cli/resolvers.py
Normal file
@@ -0,0 +1,11 @@
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def default_project_dir():
|
||||
paths = list(Path.cwd().parents)
|
||||
paths.insert(0, Path.cwd())
|
||||
return next((x for x in paths if (x / "dbt_project.yml").exists()), Path.cwd())
|
||||
|
||||
|
||||
def default_profiles_dir():
|
||||
return Path.cwd() if (Path.cwd() / "profiles.yml").exists() else Path.home() / ".dbt"
|
||||
@@ -27,6 +27,7 @@ from dbt.utils import (
|
||||
from dbt.clients._jinja_blocks import BlockIterator, BlockData, BlockTag
|
||||
from dbt.contracts.graph.compiled import CompiledGenericTestNode
|
||||
from dbt.contracts.graph.parsed import ParsedGenericTestNode
|
||||
|
||||
from dbt.exceptions import (
|
||||
InternalException,
|
||||
raise_compiler_error,
|
||||
@@ -305,13 +306,13 @@ class MacroGenerator(BaseMacroGenerator):
|
||||
@contextmanager
|
||||
def track_call(self):
|
||||
# This is only called from __call__
|
||||
if self.stack is None or self.node is None:
|
||||
if self.stack is None:
|
||||
yield
|
||||
else:
|
||||
unique_id = self.macro.unique_id
|
||||
depth = self.stack.depth
|
||||
# only mark depth=0 as a dependency
|
||||
if depth == 0:
|
||||
# only mark depth=0 as a dependency, when creating this dependency we don't pass in stack
|
||||
if depth == 0 and self.node:
|
||||
self.node.depends_on.add_macro(unique_id)
|
||||
self.stack.push(unique_id)
|
||||
try:
|
||||
|
||||
@@ -15,7 +15,7 @@ def statically_extract_macro_calls(string, ctx, db_wrapper=None):
|
||||
if hasattr(func_call, "node") and hasattr(func_call.node, "name"):
|
||||
func_name = func_call.node.name
|
||||
else:
|
||||
# func_call for dbt_utils.current_timestamp macro
|
||||
# func_call for dbt.current_timestamp macro
|
||||
# Call(
|
||||
# node=Getattr(
|
||||
# node=Name(
|
||||
|
||||
@@ -12,6 +12,7 @@ import tarfile
|
||||
import requests
|
||||
import stat
|
||||
from typing import Type, NoReturn, List, Optional, Dict, Any, Tuple, Callable, Union
|
||||
from pathspec import PathSpec # type: ignore
|
||||
|
||||
from dbt.events.functions import fire_event
|
||||
from dbt.events.types import (
|
||||
@@ -36,6 +37,7 @@ def find_matching(
|
||||
root_path: str,
|
||||
relative_paths_to_search: List[str],
|
||||
file_pattern: str,
|
||||
ignore_spec: Optional[PathSpec] = None,
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Given an absolute `root_path`, a list of relative paths to that
|
||||
@@ -57,19 +59,30 @@ def find_matching(
|
||||
reobj = re.compile(regex, re.IGNORECASE)
|
||||
|
||||
for relative_path_to_search in relative_paths_to_search:
|
||||
# potential speedup for ignore_spec
|
||||
# if ignore_spec.matches(relative_path_to_search):
|
||||
# continue
|
||||
absolute_path_to_search = os.path.join(root_path, relative_path_to_search)
|
||||
walk_results = os.walk(absolute_path_to_search)
|
||||
|
||||
for current_path, subdirectories, local_files in walk_results:
|
||||
# potential speedup for ignore_spec
|
||||
# relative_dir = os.path.relpath(current_path, root_path) + os.sep
|
||||
# if ignore_spec.match(relative_dir):
|
||||
# continue
|
||||
for local_file in local_files:
|
||||
absolute_path = os.path.join(current_path, local_file)
|
||||
relative_path = os.path.relpath(absolute_path, absolute_path_to_search)
|
||||
relative_path_to_root = os.path.join(relative_path_to_search, relative_path)
|
||||
|
||||
modification_time = 0.0
|
||||
try:
|
||||
modification_time = os.path.getmtime(absolute_path)
|
||||
except OSError:
|
||||
fire_event(SystemErrorRetrievingModTime(path=absolute_path))
|
||||
if reobj.match(local_file):
|
||||
if reobj.match(local_file) and (
|
||||
not ignore_spec or not ignore_spec.match_file(relative_path_to_root)
|
||||
):
|
||||
matching.append(
|
||||
{
|
||||
"searched_path": relative_path_to_search,
|
||||
@@ -164,7 +177,7 @@ def write_file(path: str, contents: str = "") -> bool:
|
||||
reason = "Path was possibly too long"
|
||||
# all our hard work and the path was still too long. Log and
|
||||
# continue.
|
||||
fire_event(SystemCouldNotWrite(path=path, reason=reason, exc=exc))
|
||||
fire_event(SystemCouldNotWrite(path=path, reason=reason, exc=str(exc)))
|
||||
else:
|
||||
raise
|
||||
return True
|
||||
|
||||
@@ -183,6 +183,7 @@ class Compiler:
|
||||
|
||||
context = generate_runtime_model_context(node, self.config, manifest)
|
||||
context.update(extra_context)
|
||||
|
||||
if isinstance(node, CompiledGenericTestNode):
|
||||
# for test nodes, add a special keyword args value to the context
|
||||
jinja.add_rendered_test_kwargs(context, node)
|
||||
|
||||
@@ -23,8 +23,6 @@ from .renderer import ProfileRenderer
|
||||
|
||||
DEFAULT_THREADS = 1
|
||||
|
||||
DEFAULT_PROFILES_DIR = os.path.join(os.path.expanduser("~"), ".dbt")
|
||||
|
||||
INVALID_PROFILE_MESSAGE = """
|
||||
dbt encountered an error while trying to read your profiles.yml file.
|
||||
|
||||
@@ -44,7 +42,7 @@ defined in your profiles.yml file. You can find profiles.yml here:
|
||||
|
||||
{profiles_file}/profiles.yml
|
||||
""".format(
|
||||
profiles_file=DEFAULT_PROFILES_DIR
|
||||
profiles_file=flags.DEFAULT_PROFILES_DIR
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -3,31 +3,42 @@ import os
|
||||
from copy import deepcopy
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Dict, Any, Optional, Mapping, Iterator, Iterable, Tuple, List, MutableSet, Type
|
||||
from typing import (
|
||||
Any,
|
||||
Dict,
|
||||
Iterable,
|
||||
Iterator,
|
||||
List,
|
||||
Mapping,
|
||||
MutableSet,
|
||||
Optional,
|
||||
Tuple,
|
||||
Type,
|
||||
Union,
|
||||
)
|
||||
|
||||
from .profile import Profile
|
||||
from .project import Project
|
||||
from .renderer import DbtProjectYamlRenderer, ProfileRenderer
|
||||
from .utils import parse_cli_vars
|
||||
from dbt import flags
|
||||
from dbt.adapters.factory import get_relation_class_by_name, get_include_paths
|
||||
from dbt.helper_types import FQNPath, PathSet, DictDefaultEmptyStr
|
||||
from dbt.adapters.factory import get_include_paths, get_relation_class_by_name
|
||||
from dbt.config.profile import read_user_config
|
||||
from dbt.contracts.connection import AdapterRequiredConfig, Credentials
|
||||
from dbt.contracts.graph.manifest import ManifestMetadata
|
||||
from dbt.contracts.relation import ComponentName
|
||||
from dbt.ui import warning_tag
|
||||
|
||||
from dbt.contracts.project import Configuration, UserConfig
|
||||
from dbt.contracts.relation import ComponentName
|
||||
from dbt.dataclass_schema import ValidationError
|
||||
from dbt.exceptions import (
|
||||
RuntimeException,
|
||||
DbtProjectError,
|
||||
RuntimeException,
|
||||
raise_compiler_error,
|
||||
validator_error_message,
|
||||
warn_or_error,
|
||||
raise_compiler_error,
|
||||
)
|
||||
from dbt.helper_types import DictDefaultEmptyStr, FQNPath, PathSet
|
||||
from dbt.ui import warning_tag
|
||||
|
||||
from dbt.dataclass_schema import ValidationError
|
||||
from .profile import Profile
|
||||
from .project import Project, PartialProject
|
||||
from .renderer import DbtProjectYamlRenderer, ProfileRenderer
|
||||
from .utils import parse_cli_vars
|
||||
|
||||
|
||||
def _project_quoting_dict(proj: Project, profile: Profile) -> Dict[ComponentName, bool]:
|
||||
@@ -190,28 +201,52 @@ class RuntimeConfig(Project, Profile, AdapterRequiredConfig):
|
||||
|
||||
@classmethod
|
||||
def collect_parts(cls: Type["RuntimeConfig"], args: Any) -> Tuple[Project, Profile]:
|
||||
# profile_name from the project
|
||||
project_root = args.project_dir if args.project_dir else os.getcwd()
|
||||
version_check = bool(flags.VERSION_CHECK)
|
||||
partial = Project.partial_load(project_root, verify_version=version_check)
|
||||
|
||||
# build the profile using the base renderer and the one fact we know
|
||||
# Note: only the named profile section is rendered. The rest of the
|
||||
# profile is ignored.
|
||||
cli_vars: Dict[str, Any] = parse_cli_vars(getattr(args, "vars", "{}"))
|
||||
|
||||
profile = cls.collect_profile(args=args)
|
||||
project_renderer = DbtProjectYamlRenderer(profile, cli_vars)
|
||||
project = cls.collect_project(args=args, project_renderer=project_renderer)
|
||||
assert type(project) is Project
|
||||
return (project, profile)
|
||||
|
||||
@classmethod
|
||||
def collect_profile(
|
||||
cls: Type["RuntimeConfig"], args: Any, profile_name: Optional[str] = None
|
||||
) -> Profile:
|
||||
|
||||
cli_vars: Dict[str, Any] = parse_cli_vars(getattr(args, "vars", "{}"))
|
||||
profile_renderer = ProfileRenderer(cli_vars)
|
||||
profile_name = partial.render_profile_name(profile_renderer)
|
||||
|
||||
# build the profile using the base renderer and the one fact we know
|
||||
if profile_name is None:
|
||||
# Note: only the named profile section is rendered here. The rest of the
|
||||
# profile is ignored.
|
||||
partial = cls.collect_project(args)
|
||||
assert type(partial) is PartialProject
|
||||
profile_name = partial.render_profile_name(profile_renderer)
|
||||
|
||||
profile = cls._get_rendered_profile(args, profile_renderer, profile_name)
|
||||
# Save env_vars encountered in rendering for partial parsing
|
||||
profile.profile_env_vars = profile_renderer.ctx_obj.env_vars
|
||||
return profile
|
||||
|
||||
# get a new renderer using our target information and render the
|
||||
# project
|
||||
project_renderer = DbtProjectYamlRenderer(profile, cli_vars)
|
||||
project = partial.render(project_renderer)
|
||||
# Save env_vars encountered in rendering for partial parsing
|
||||
project.project_env_vars = project_renderer.ctx_obj.env_vars
|
||||
return (project, profile)
|
||||
@classmethod
|
||||
def collect_project(
|
||||
cls: Type["RuntimeConfig"],
|
||||
args: Any,
|
||||
project_renderer: Optional[DbtProjectYamlRenderer] = None,
|
||||
) -> Union[Project, PartialProject]:
|
||||
|
||||
project_root = args.project_dir if args.project_dir else os.getcwd()
|
||||
version_check = bool(flags.VERSION_CHECK)
|
||||
partial = Project.partial_load(project_root, verify_version=version_check)
|
||||
if project_renderer is None:
|
||||
return partial
|
||||
else:
|
||||
project = partial.render(project_renderer)
|
||||
project.project_env_vars = project_renderer.ctx_obj.env_vars
|
||||
return project
|
||||
|
||||
# Called in main.py, lib.py, task/base.py
|
||||
@classmethod
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user