Compare commits

..

1 Commits

Author SHA1 Message Date
Jeremy Cohen
93572b9291 Support 'scripts' for python models 2022-09-28 20:25:53 +02:00
651 changed files with 16160 additions and 32158 deletions

View File

@@ -1,15 +1,13 @@
[bumpversion]
current_version = 1.4.2
parse = (?P<major>[\d]+) # major version number
\.(?P<minor>[\d]+) # minor version number
\.(?P<patch>[\d]+) # patch version number
(((?P<prekind>a|b|rc) # optional pre-release type
?(?P<num>[\d]+?)) # optional pre-release version number
\.?(?P<nightly>[a-z0-9]+\+[a-z]+)? # optional nightly release indicator
current_version = 1.3.0b2
parse = (?P<major>\d+)
\.(?P<minor>\d+)
\.(?P<patch>\d+)
((?P<prekind>a|b|rc)
(?P<pre>\d+) # pre-release version num
)?
serialize =
{major}.{minor}.{patch}{prekind}{num}.{nightly}
{major}.{minor}.{patch}{prekind}{num}
{major}.{minor}.{patch}{prekind}{pre}
{major}.{minor}.{patch}
commit = False
tag = False
@@ -23,11 +21,9 @@ values =
rc
final
[bumpversion:part:num]
[bumpversion:part:pre]
first_value = 1
[bumpversion:part:nightly]
[bumpversion:file:core/setup.py]
[bumpversion:file:core/dbt/version.py]

View File

@@ -3,7 +3,6 @@
For information on prior major and minor releases, see their changelogs:
* [1.3](https://github.com/dbt-labs/dbt-core/blob/1.3.latest/CHANGELOG.md)
* [1.2](https://github.com/dbt-labs/dbt-core/blob/1.2.latest/CHANGELOG.md)
* [1.1](https://github.com/dbt-labs/dbt-core/blob/1.1.latest/CHANGELOG.md)
* [1.0](https://github.com/dbt-labs/dbt-core/blob/1.0.latest/CHANGELOG.md)

38
.changes/1.3.0-b1.md Normal file
View File

@@ -0,0 +1,38 @@
## dbt-core 1.3.0-b1 - July 29, 2022
### Features
- Python model inital version ([#5261](https://github.com/dbt-labs/dbt-core/issues/5261), [#5421](https://github.com/dbt-labs/dbt-core/pull/5421))
- allows user to include the file extension for .py models in the dbt run -m command. ([#5289](https://github.com/dbt-labs/dbt-core/issues/5289), [#5295](https://github.com/dbt-labs/dbt-core/pull/5295))
- Incremental materialization refactor and cleanup ([#5245](https://github.com/dbt-labs/dbt-core/issues/5245), [#5359](https://github.com/dbt-labs/dbt-core/pull/5359))
- Python models can support incremental logic ([#0](https://github.com/dbt-labs/dbt-core/issues/0), [#35](https://github.com/dbt-labs/dbt-core/pull/35))
- Add reusable function for retrying adapter connections. Utilize said function to add retries for Postgres (and Redshift). ([#5022](https://github.com/dbt-labs/dbt-core/issues/5022), [#5432](https://github.com/dbt-labs/dbt-core/pull/5432))
- add exponential backoff to connection retries on Postgres (and Redshift) ([#5502](https://github.com/dbt-labs/dbt-core/issues/5502), [#5503](https://github.com/dbt-labs/dbt-core/pull/5503))
### Fixes
- Add context to compilation errors generated while rendering generic test configuration values. ([#5294](https://github.com/dbt-labs/dbt-core/issues/5294), [#5393](https://github.com/dbt-labs/dbt-core/pull/5393))
- Rename try to strict for more intuitiveness ([#5475](https://github.com/dbt-labs/dbt-core/issues/5475), [#5477](https://github.com/dbt-labs/dbt-core/pull/5477))
- Ignore empty strings passed in as secrets ([#5312](https://github.com/dbt-labs/dbt-core/issues/5312), [#5518](https://github.com/dbt-labs/dbt-core/pull/5518))
- Fix handling of top-level exceptions ([#5564](https://github.com/dbt-labs/dbt-core/issues/5564), [#5560](https://github.com/dbt-labs/dbt-core/pull/5560))
### Docs
- Update dependency inline-source from ^6.1.5 to ^7.2.0 ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
- Update dependency jest from ^26.2.2 to ^28.1.3 ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
- Update dependency underscore from ^1.9.0 to ^1.13.4 ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
- Update dependency webpack-cli from ^3.3.12 to ^4.7.0 ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
- Update dependency webpack-dev-server from ^3.1.11 to ^4.9.3 ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
- Searches no longer require perfect matches, and instead consider each word individually. `my model` or `model my` will now find `my_model`, without the need for underscores ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
- Support the renaming of SQL to code happening in dbt-core ([#5574](https://github.com/dbt-labs/dbt-core/issues/5574), [#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
### Under the Hood
- Added language to tracked fields in run_model event ([#5571](https://github.com/dbt-labs/dbt-core/issues/5571), [#5469](https://github.com/dbt-labs/dbt-core/pull/5469))
- Update mashumaro to 3.0.3 ([#4940](https://github.com/dbt-labs/dbt-core/issues/4940), [#5118](https://github.com/dbt-labs/dbt-core/pull/5118))
- Add python incremental materialization test ([#0000](https://github.com/dbt-labs/dbt-core/issues/0000), [#5571](https://github.com/dbt-labs/dbt-core/pull/5571))
### Dependencies
- Upgrade to Jinja2==3.1.2 from Jinja2==2.11.3 ([#4748](https://github.com/dbt-labs/dbt-core/issues/4748), [#5465](https://github.com/dbt-labs/dbt-core/pull/5465))
- Bump mypy from 0.961 to 0.971 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5495](https://github.com/dbt-labs/dbt-core/pull/5495))
- Remove pin for MarkUpSafe from >=0.23,<2.1 ([#5506](https://github.com/dbt-labs/dbt-core/issues/5506), [#5507](https://github.com/dbt-labs/dbt-core/pull/5507))
### Contributors
- [@Goodkat](https://github.com/Goodkat) ([#5518](https://github.com/dbt-labs/dbt-core/pull/5518))
- [@drewbanin](https://github.com/drewbanin) ([#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
- [@jeremyyeo](https://github.com/jeremyyeo) ([#5477](https://github.com/dbt-labs/dbt-core/pull/5477))
- [@joellabes](https://github.com/joellabes) ([#5577](https://github.com/dbt-labs/dbt-core/pull/5577))
- [@leoebfolsom](https://github.com/leoebfolsom) ([#5295](https://github.com/dbt-labs/dbt-core/pull/5295))
- [@nicholasyager](https://github.com/nicholasyager) ([#5393](https://github.com/dbt-labs/dbt-core/pull/5393))
- [@tomasfarias](https://github.com/tomasfarias) ([#5432](https://github.com/dbt-labs/dbt-core/pull/5432))

49
.changes/1.3.0-b2.md Normal file
View File

@@ -0,0 +1,49 @@
## dbt-core 1.3.0-b2 - August 29, 2022
### Features
- Add `--defer` flag to dbt compile & dbt docs generate ([#4110](https://github.com/dbt-labs/dbt-core/issues/4110), [#4514](https://github.com/dbt-labs/dbt-core/pull/4514))
- use MethodName.File when value ends with .csv ([#5578](https://github.com/dbt-labs/dbt-core/issues/5578), [#5581](https://github.com/dbt-labs/dbt-core/pull/5581))
- Make `docs` configurable in `dbt_project.yml` and add a `node_color` attribute to change the color of nodes in the DAG ([#5333](https://github.com/dbt-labs/dbt-core/issues/5333), [#5397](https://github.com/dbt-labs/dbt-core/pull/5397))
- Adding ResolvedMetricReference helper functions and tests ([#5567](https://github.com/dbt-labs/dbt-core/issues/5567), [#5607](https://github.com/dbt-labs/dbt-core/pull/5607))
- Check dbt-core version requirements when installing Hub packages ([#5648](https://github.com/dbt-labs/dbt-core/issues/5648), [#5651](https://github.com/dbt-labs/dbt-core/pull/5651))
### Fixes
- Remove the default 256 characters limit on postgres character varying type when no limitation is set ([#5238](https://github.com/dbt-labs/dbt-core/issues/5238), [#5292](https://github.com/dbt-labs/dbt-core/pull/5292))
- Include schema file config in unrendered_config ([#5338](https://github.com/dbt-labs/dbt-core/issues/5338), [#5344](https://github.com/dbt-labs/dbt-core/pull/5344))
- Resolves #5351 - Do not consider shorter varchar cols as schema changes ([#5351](https://github.com/dbt-labs/dbt-core/issues/5351), [#5395](https://github.com/dbt-labs/dbt-core/pull/5395))
- on_shchma_change fail verbosity enhancement ([#5504](https://github.com/dbt-labs/dbt-core/issues/5504), [#5505](https://github.com/dbt-labs/dbt-core/pull/5505))
- Fix error rendering docs block in metrics description ([#5585](https://github.com/dbt-labs/dbt-core/issues/5585), [#5603](https://github.com/dbt-labs/dbt-core/pull/5603))
- Extended validations for the project names ([#5379](https://github.com/dbt-labs/dbt-core/issues/5379), [#5620](https://github.com/dbt-labs/dbt-core/pull/5620))
- Use sys.exit instead of exit ([#5621](https://github.com/dbt-labs/dbt-core/issues/5621), [#5627](https://github.com/dbt-labs/dbt-core/pull/5627))
- Finishing logic upgrade to Redshift for name truncation collisions. ([#5586](https://github.com/dbt-labs/dbt-core/issues/5586), [#5656](https://github.com/dbt-labs/dbt-core/pull/5656))
- multiple args for ref and source ([#5634](https://github.com/dbt-labs/dbt-core/issues/5634), [#5635](https://github.com/dbt-labs/dbt-core/pull/5635))
- Fix Unexpected behavior when chaining methods on dbt-ref'ed/sourced dataframes ([#5646](https://github.com/dbt-labs/dbt-core/issues/5646), [#5677](https://github.com/dbt-labs/dbt-core/pull/5677))
### Docs
- Leverages `docs.node_color` from `dbt-core` to color nodes in the DAG ([dbt-docs/#44](https://github.com/dbt-labs/dbt-docs/issues/44), [dbt-docs/#281](https://github.com/dbt-labs/dbt-docs/pull/281))
### Under the Hood
- Save use of default env vars to manifest to enable partial parsing in those cases. ([#5155](https://github.com/dbt-labs/dbt-core/issues/5155), [#5589](https://github.com/dbt-labs/dbt-core/pull/5589))
- add more information to log line interop test failures ([#5658](https://github.com/dbt-labs/dbt-core/issues/5658), [#5659](https://github.com/dbt-labs/dbt-core/pull/5659))
- Add supported languages to materializations ([#5569](https://github.com/dbt-labs/dbt-core/issues/5569), [#5695](https://github.com/dbt-labs/dbt-core/pull/5695))
### Dependency
- Bump python from 3.10.5-slim-bullseye to 3.10.6-slim-bullseye in /docker ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5623](https://github.com/dbt-labs/dbt-core/pull/5623))
- Bump mashumaro[msgpack] from 3.0.3 to 3.0.4 in /core ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5649](https://github.com/dbt-labs/dbt-core/pull/5649))
### Contributors
- [@Goodkat](https://github.com/Goodkat) ([#5581](https://github.com/dbt-labs/dbt-core/pull/5581), [#5620](https://github.com/dbt-labs/dbt-core/pull/5620))
- [@Ilanbenb](https://github.com/Ilanbenb) ([#5505](https://github.com/dbt-labs/dbt-core/pull/5505))
- [@b-per](https://github.com/b-per) ([#5397](https://github.com/dbt-labs/dbt-core/pull/5397), [dbt-docs/#281](https://github.com/dbt-labs/dbt-docs/pull/281))
- [@callum-mcdata](https://github.com/callum-mcdata) ([#5607](https://github.com/dbt-labs/dbt-core/pull/5607))
- [@epapineau](https://github.com/epapineau) ([#5395](https://github.com/dbt-labs/dbt-core/pull/5395))
- [@kadero](https://github.com/kadero) ([#4514](https://github.com/dbt-labs/dbt-core/pull/4514))
- [@matt-winkler](https://github.com/matt-winkler) ([#5397](https://github.com/dbt-labs/dbt-core/pull/5397), [dbt-docs/#281](https://github.com/dbt-labs/dbt-docs/pull/281))
- [@shrodingers](https://github.com/shrodingers) ([#5292](https://github.com/dbt-labs/dbt-core/pull/5292))
- [@sungchun12](https://github.com/sungchun12) ([#5397](https://github.com/dbt-labs/dbt-core/pull/5397), [dbt-docs/#281](https://github.com/dbt-labs/dbt-docs/pull/281))
- [@varun-dc](https://github.com/varun-dc) ([#5627](https://github.com/dbt-labs/dbt-core/pull/5627))

View File

@@ -0,0 +1,7 @@
kind: Dependencies
body: Upgrade to Jinja2==3.1.2 from Jinja2==2.11.3
time: 2022-07-19T14:16:46.665631-05:00
custom:
Author: emmyoop
Issue: "4748"
PR: "5465"

View File

@@ -0,0 +1,7 @@
kind: Dependencies
body: "Bump mypy from 0.961 to 0.971"
time: 2022-07-20T00:07:41.000000-05:00
custom:
Author: dependabot[bot]
Issue: "4904"
PR: "5495"

View File

@@ -0,0 +1,7 @@
kind: Dependencies
body: Remove pin for MarkUpSafe from >=0.23,<2.1
time: 2022-07-21T09:32:33.494002-05:00
custom:
Author: emmyoop
Issue: "5506"
PR: "5507"

View File

@@ -0,0 +1,7 @@
kind: Dependency
body: "Bump python from 3.10.5-slim-bullseye to 3.10.6-slim-bullseye in /docker"
time: 2022-08-08T13:23:27.00000Z
custom:
Author: dependabot[bot]
Issue: 4904
PR: 5623

View File

@@ -0,0 +1,7 @@
kind: Dependency
body: "Bump mashumaro[msgpack] from 3.0.3 to 3.0.4 in /core"
time: 2022-08-15T18:27:31.00000Z
custom:
Author: dependabot[bot]
Issue: 4904
PR: 5649

View File

@@ -0,0 +1,7 @@
kind: Docs
body: Update dependency inline-source from ^6.1.5 to ^7.2.0
time: 2022-07-28T14:02:58.441963-07:00
custom:
Author: emmyoop
Issue: "299"
PR: "291"

View File

@@ -0,0 +1,7 @@
kind: Docs
body: Update dependency jest from ^26.2.2 to ^28.1.3
time: 2022-07-28T14:03:29.837274-07:00
custom:
Author: emmyoop
Issue: "299"
PR: "291"

View File

@@ -0,0 +1,7 @@
kind: Docs
body: Update dependency underscore from ^1.9.0 to ^1.13.4
time: 2022-07-28T14:03:51.123441-07:00
custom:
Author: emmyoop
Issue: "299"
PR: "291"

View File

@@ -0,0 +1,7 @@
kind: Docs
body: Update dependency webpack-cli from ^3.3.12 to ^4.7.0
time: 2022-07-28T14:04:25.629638-07:00
custom:
Author: emmyoop
Issue: "299"
PR: "291"

View File

@@ -0,0 +1,7 @@
kind: Docs
body: Update dependency webpack-dev-server from ^3.1.11 to ^4.9.3
time: 2022-07-28T14:04:49.637369-07:00
custom:
Author: emmyoop
Issue: "299"
PR: "291"

View File

@@ -0,0 +1,8 @@
kind: Docs
body: Searches no longer require perfect matches, and instead consider each word individually.
`my model` or `model my` will now find `my_model`, without the need for underscores
time: 2022-07-28T14:06:20.371364-07:00
custom:
Author: joellabes
Issue: "143"
PR: "145"

View File

@@ -0,0 +1,7 @@
kind: Docs
body: Support the renaming of SQL to code happening in dbt-core
time: 2022-07-28T14:08:06.184934-07:00
custom:
Author: jtcohen6 stu-k drewbanin ChenyuLInx
Issue: "299"
PR: "292"

View File

@@ -0,0 +1,7 @@
kind: Docs
body: Leverages `docs.node_color` from `dbt-core` to color nodes in the DAG
time: 2022-08-04T13:41:38.669987-05:00
custom:
Author: matt-winkler sungchun12 b-per
Issue: "44"
PR: "281"

View File

@@ -0,0 +1,7 @@
kind: Features
body: Add `--defer` flag to dbt compile & dbt docs generate
time: 2022-03-14T18:11:27.783107864+01:00
custom:
Author: kadero
Issue: "4110"
PR: "4514"

View File

@@ -0,0 +1,7 @@
kind: Features
body: Python model inital version
time: 2022-05-10T16:51:30.245589-07:00
custom:
Author: ChenyuLInx
Issue: "5261"
PR: "5421"

View File

@@ -0,0 +1,7 @@
kind: Features
body: allows user to include the file extension for .py models in the dbt run -m command.
time: 2022-05-26T16:53:23.389671-07:00
custom:
Author: leoebfolsom
Issue: "5289"
PR: "5295"

View File

@@ -0,0 +1,7 @@
kind: Features
body: Incremental materialization refactor and cleanup
time: 2022-06-10T10:56:47.226887-04:00
custom:
Author: gshank
Issue: "5245"
PR: "5359"

View File

@@ -0,0 +1,7 @@
kind: Features
body: Python models can support incremental logic
time: 2022-06-27T13:10:42.123303-05:00
custom:
Author: iknox-fa
Issue: "0"
PR: "35"

View File

@@ -0,0 +1,8 @@
kind: Features
body: Add reusable function for retrying adapter connections. Utilize said function
to add retries for Postgres (and Redshift).
time: 2022-07-15T03:55:55.270637265+02:00
custom:
Author: tomasfarias
Issue: "5022"
PR: "5432"

View File

@@ -0,0 +1,7 @@
kind: Features
body: add exponential backoff to connection retries on Postgres (and Redshift)
time: 2022-07-20T17:12:57.486949-04:00
custom:
Author: nathaniel-may
Issue: "5502"
PR: "5503"

View File

@@ -0,0 +1,7 @@
kind: Features
body: use MethodName.File when value ends with .csv
time: 2022-07-29T17:32:31.395677157+02:00
custom:
Author: Goodkat
Issue: "5578"
PR: "5581"

View File

@@ -0,0 +1,8 @@
kind: Features
body: Make `docs` configurable in `dbt_project.yml` and add a `node_color` attribute
to change the color of nodes in the DAG
time: 2022-08-03T10:42:30.60624+02:00
custom:
Author: matt-winkler sungchun12 b-per
Issue: "5333"
PR: "5397"

View File

@@ -0,0 +1,7 @@
kind: Features
body: Adding ResolvedMetricReference helper functions and tests
time: 2022-08-04T12:09:36.202919-04:00
custom:
Author: callum-mcdata
Issue: "5567"
PR: "5607"

View File

@@ -0,0 +1,7 @@
kind: Features
body: Check dbt-core version requirements when installing Hub packages
time: 2022-08-15T13:43:12.965143+01:00
custom:
Author: jtcohen6
Issue: "5648"
PR: "5651"

View File

@@ -0,0 +1,8 @@
kind: Fixes
body: Remove the default 256 characters limit on postgres character varying type when
no limitation is set
time: 2022-05-23T10:38:43.392232+02:00
custom:
Author: shrodingers
Issue: "5238"
PR: "5292"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: Include schema file config in unrendered_config
time: 2022-06-07T12:30:58.535207-04:00
custom:
Author: gshank
Issue: "5338"
PR: "5344"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: 'Resolves #5351 - Do not consider shorter varchar cols as schema changes'
time: 2022-06-17T19:37:31.885484-07:00
custom:
Author: epapineau
Issue: "5351"
PR: "5395"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: Rename try to strict for more intuitiveness
time: 2022-07-15T23:11:48.327928+12:00
custom:
Author: jeremyyeo
Issue: "5475"
PR: "5477"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: on_shchma_change fail verbosity enhancement
time: 2022-07-20T21:37:46.474241+03:00
custom:
Author: Ilanbenb
Issue: "5504"
PR: "5505"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: Ignore empty strings passed in as secrets
time: 2022-07-23T21:53:30.907759094+02:00
custom:
Author: Goodkat
Issue: "5312"
PR: "5518"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: Fix handling of top-level exceptions
time: 2022-07-26T11:36:36.824979-04:00
custom:
Author: gshank
Issue: "5564"
PR: "5560"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: Fix error rendering docs block in metrics description
time: 2022-08-03T14:42:21.386265-04:00
custom:
Author: gshank
Issue: "5585"
PR: "5603"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: Extended validations for the project names
time: 2022-08-05T22:10:22.746830854+02:00
custom:
Author: Goodkat
Issue: "5379"
PR: "5620"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: Use sys.exit instead of exit
time: 2022-08-08T11:20:01.838171926-04:00
custom:
Author: varun-dc
Issue: "5621"
PR: "5627"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: Finishing logic upgrade to Redshift for name truncation collisions.
time: 2022-08-15T23:04:09.173645-07:00
custom:
Author: versusfacit
Issue: "5586"
PR: "5656"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: multiple args for ref and source
time: 2022-08-16T15:34:01.348339-07:00
custom:
Author: ChenyuLInx
Issue: "5634"
PR: "5635"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: Fix Unexpected behavior when chaining methods on dbt-ref'ed/sourced dataframes
time: 2022-08-17T16:36:42.678275-07:00
custom:
Author: ChenyuLInx
Issue: "5646"
PR: "5677"

View File

@@ -0,0 +1,8 @@
kind: Fixes
body: Add context to compilation errors generated while rendering generic test configuration
values.
time: 2022-06-17T15:07:44.751037-04:00
custom:
Author: nicholasyager
Issue: "5294"
PR: "5393"

View File

@@ -0,0 +1,7 @@
kind: Under the Hood
body: Added language to tracked fields in run_model event
time: 2022-07-13T12:49:25.362678-05:00
custom:
Author: stu-k
Issue: "5571"
PR: "5469"

View File

@@ -0,0 +1,7 @@
kind: Under the Hood
body: Update mashumaro to 3.0.3
time: 2022-07-20T11:52:26.210876-04:00
custom:
Author: gshank
Issue: "4940"
PR: "5118"

View File

@@ -0,0 +1,7 @@
kind: Under the Hood
body: Add python incremental materialization test
time: 2022-07-28T09:45:36.13608-05:00
custom:
Author: stu-k
Issue: "0000"
PR: "5571"

View File

@@ -0,0 +1,8 @@
kind: Under the Hood
body: Save use of default env vars to manifest to enable partial parsing in those
cases.
time: 2022-08-02T11:29:36.417589-05:00
custom:
Author: emmyoop
Issue: "5155"
PR: "5589"

View File

@@ -0,0 +1,7 @@
kind: Under the Hood
body: add more information to log line interop test failures
time: 2022-08-16T12:20:32.119588+01:00
custom:
Author: nathaniel-may
Issue: "5658"
PR: "5659"

View File

@@ -0,0 +1,7 @@
kind: Under the Hood
body: Add supported languages to materializations
time: 2022-08-22T10:37:39.50743-05:00
custom:
Author: stu-k
Issue: "5569"
PR: "5695"

View File

@@ -1,126 +0,0 @@
## dbt-core 1.4.0 - January 25, 2023
### Breaking Changes
- Cleaned up exceptions to directly raise in code. Also updated the existing exception to meet PEP guidelines.Removed use of all exception functions in the code base and marked them all as deprecated to be removed next minor release. ([#6339](https://github.com/dbt-labs/dbt-core/issues/6339), [#6393](https://github.com/dbt-labs/dbt-core/issues/6393), [#6460](https://github.com/dbt-labs/dbt-core/issues/6460))
### Features
- Added favor-state flag to optionally favor state nodes even if unselected node exists ([#5016](https://github.com/dbt-labs/dbt-core/issues/5016))
- Update structured logging. Convert to using protobuf messages. Ensure events are enriched with node_info. ([#5610](https://github.com/dbt-labs/dbt-core/issues/5610))
- incremental predicates ([#5680](https://github.com/dbt-labs/dbt-core/issues/5680))
- Friendlier error messages when packages.yml is malformed ([#5486](https://github.com/dbt-labs/dbt-core/issues/5486))
- Allow partitions in external tables to be supplied as a list ([#5929](https://github.com/dbt-labs/dbt-core/issues/5929))
- extend -f flag shorthand for seed command ([#5990](https://github.com/dbt-labs/dbt-core/issues/5990))
- This pulls the profile name from args when constructing a RuntimeConfig in lib.py, enabling the dbt-server to override the value that's in the dbt_project.yml ([#6201](https://github.com/dbt-labs/dbt-core/issues/6201))
- Adding tarball install method for packages. Allowing package tarball to be specified via url in the packages.yaml. ([#4205](https://github.com/dbt-labs/dbt-core/issues/4205))
- Added an md5 function to the base context ([#6246](https://github.com/dbt-labs/dbt-core/issues/6246))
- Exposures support metrics in lineage ([#6057](https://github.com/dbt-labs/dbt-core/issues/6057))
- Add support for Python 3.11 ([#6147](https://github.com/dbt-labs/dbt-core/issues/6147))
- Making timestamp optional for metrics ([#6398](https://github.com/dbt-labs/dbt-core/issues/6398))
- The meta configuration field is now included in the node_info property of structured logs. ([#6216](https://github.com/dbt-labs/dbt-core/issues/6216))
- Adds buildable selection mode ([#6365](https://github.com/dbt-labs/dbt-core/issues/6365))
- --warn-error-options: Treat warnings as errors for specific events, based on user configuration ([#6165](https://github.com/dbt-labs/dbt-core/issues/6165))
### Fixes
- Account for disabled flags on models in schema files more completely ([#3992](https://github.com/dbt-labs/dbt-core/issues/3992))
- Add validation of enabled config for metrics, exposures and sources ([#6030](https://github.com/dbt-labs/dbt-core/issues/6030))
- check length of args of python model function before accessing it ([#6041](https://github.com/dbt-labs/dbt-core/issues/6041))
- Add functors to ensure event types with str-type attributes are initialized to spec, even when provided non-str type params. ([#5436](https://github.com/dbt-labs/dbt-core/issues/5436))
- Allow hooks to fail without halting execution flow ([#5625](https://github.com/dbt-labs/dbt-core/issues/5625))
- fix missing f-strings, convert old .format() messages to f-strings for consistency ([#6241](https://github.com/dbt-labs/dbt-core/issues/6241))
- Clarify Error Message for how many models are allowed in a Python file ([#6245](https://github.com/dbt-labs/dbt-core/issues/6245))
- Fix typo in util.py ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904))
- After this, will be possible to use default values for dbt.config.get ([#6309](https://github.com/dbt-labs/dbt-core/issues/6309))
- Use full path for writing manifest ([#6055](https://github.com/dbt-labs/dbt-core/issues/6055))
- add pre-commit install to make dev script in Makefile ([#6269](https://github.com/dbt-labs/dbt-core/issues/6269))
- Late-rendering for `pre_` and `post_hook`s in `dbt_project.yml` ([#6411](https://github.com/dbt-labs/dbt-core/issues/6411))
- [CT-1284] Change Python model default materialization to table ([#5989](https://github.com/dbt-labs/dbt-core/issues/5989))
- [CT-1591] Don't parse empty Python files ([#6345](https://github.com/dbt-labs/dbt-core/issues/6345))
- Repair a regression which prevented basic logging before the logging subsystem is completely configured. ([#6434](https://github.com/dbt-labs/dbt-core/issues/6434))
- fix docs generate --defer by adding defer_to_manifest to before_run ([#6488](https://github.com/dbt-labs/dbt-core/issues/6488))
- Bug when partial parsing with an empty schema file ([#4850](https://github.com/dbt-labs/dbt-core/issues/4850))
- Fix DBT_FAVOR_STATE env var ([#5859](https://github.com/dbt-labs/dbt-core/issues/5859))
- Restore historical behavior of certain disabled test messages, so that they are at the less obtrusive debug level, rather than the warning level. ([#6501](https://github.com/dbt-labs/dbt-core/issues/6501))
- Bump mashumuro version to get regression fix and add unit test to verify that fix. ([#6428](https://github.com/dbt-labs/dbt-core/issues/6428))
- Call update_event_status earlier for node results. Rename event 'HookFinished' -> FinishedRunningStats ([#6571](https://github.com/dbt-labs/dbt-core/issues/6571))
- Provide backward compatibility for `get_merge_sql` arguments ([#6625](https://github.com/dbt-labs/dbt-core/issues/6625))
- Fix behavior of --favor-state with --defer ([#6617](https://github.com/dbt-labs/dbt-core/issues/6617))
- Include adapter_response in NodeFinished run_result log event ([#6703](https://github.com/dbt-labs/dbt-core/issues/6703))
### Docs
- minor doc correction ([dbt-docs/#5791](https://github.com/dbt-labs/dbt-docs/issues/5791))
- Generate API docs for new CLI interface ([dbt-docs/#5528](https://github.com/dbt-labs/dbt-docs/issues/5528))
- ([dbt-docs/#5880](https://github.com/dbt-labs/dbt-docs/issues/5880))
- Fix rendering of sample code for metrics ([dbt-docs/#323](https://github.com/dbt-labs/dbt-docs/issues/323))
- Alphabetize `core/dbt/README.md` ([dbt-docs/#6368](https://github.com/dbt-labs/dbt-docs/issues/6368))
- Updated minor typos encountered when skipping profile setup ([dbt-docs/#6529](https://github.com/dbt-labs/dbt-docs/issues/6529))
### Under the Hood
- Put black config in explicit config ([#5946](https://github.com/dbt-labs/dbt-core/issues/5946))
- Added flat_graph attribute the Manifest class's deepcopy() coverage ([#5809](https://github.com/dbt-labs/dbt-core/issues/5809))
- Add mypy configs so `mypy` passes from CLI ([#5983](https://github.com/dbt-labs/dbt-core/issues/5983))
- Exception message cleanup. ([#6023](https://github.com/dbt-labs/dbt-core/issues/6023))
- Add dmypy cache to gitignore ([#6028](https://github.com/dbt-labs/dbt-core/issues/6028))
- Provide useful errors when the value of 'materialized' is invalid ([#5229](https://github.com/dbt-labs/dbt-core/issues/5229))
- Clean up string formatting ([#6068](https://github.com/dbt-labs/dbt-core/issues/6068))
- Fixed extra whitespace in strings introduced by black. ([#1350](https://github.com/dbt-labs/dbt-core/issues/1350))
- Remove the 'root_path' field from most nodes ([#6171](https://github.com/dbt-labs/dbt-core/issues/6171))
- Combine certain logging events with different levels ([#6173](https://github.com/dbt-labs/dbt-core/issues/6173))
- Convert threading tests to pytest ([#5942](https://github.com/dbt-labs/dbt-core/issues/5942))
- Convert postgres index tests to pytest ([#5770](https://github.com/dbt-labs/dbt-core/issues/5770))
- Convert use color tests to pytest ([#5771](https://github.com/dbt-labs/dbt-core/issues/5771))
- Add github actions workflow to generate high level CLI API docs ([#5942](https://github.com/dbt-labs/dbt-core/issues/5942))
- Functionality-neutral refactor of event logging system to improve encapsulation and modularity. ([#6139](https://github.com/dbt-labs/dbt-core/issues/6139))
- Consolidate ParsedNode and CompiledNode classes ([#6383](https://github.com/dbt-labs/dbt-core/issues/6383))
- Prevent doc gen workflow from running on forks ([#6386](https://github.com/dbt-labs/dbt-core/issues/6386))
- Fix intermittent database connection failure in Windows CI test ([#6394](https://github.com/dbt-labs/dbt-core/issues/6394))
- Refactor and clean up manifest nodes ([#6426](https://github.com/dbt-labs/dbt-core/issues/6426))
- Restore important legacy logging behaviors, following refactor which removed them ([#6437](https://github.com/dbt-labs/dbt-core/issues/6437))
- Treat dense text blobs as binary for `git grep` ([#6294](https://github.com/dbt-labs/dbt-core/issues/6294))
- Prune partial parsing logging events ([#6313](https://github.com/dbt-labs/dbt-core/issues/6313))
- Updating the deprecation warning in the metric attributes renamed event ([#6507](https://github.com/dbt-labs/dbt-core/issues/6507))
- [CT-1693] Port severity test to Pytest ([#6466](https://github.com/dbt-labs/dbt-core/issues/6466))
- [CT-1694] Deprecate event tracking tests ([#6467](https://github.com/dbt-labs/dbt-core/issues/6467))
- Reorganize structured logging events to have two top keys ([#6311](https://github.com/dbt-labs/dbt-core/issues/6311))
- Combine some logging events ([#1716](https://github.com/dbt-labs/dbt-core/issues/1716), [#1717](https://github.com/dbt-labs/dbt-core/issues/1717), [#1719](https://github.com/dbt-labs/dbt-core/issues/1719))
- Check length of escaped strings in the adapter test ([#6566](https://github.com/dbt-labs/dbt-core/issues/6566))
### Dependencies
- Update pathspec requirement from ~=0.9.0 to >=0.9,<0.11 in /core ([#5917](https://github.com/dbt-labs/dbt-core/pull/5917))
- Bump black from 22.8.0 to 22.10.0 ([#6019](https://github.com/dbt-labs/dbt-core/pull/6019))
- Bump mashumaro[msgpack] from 3.0.4 to 3.1.1 in /core ([#6108](https://github.com/dbt-labs/dbt-core/pull/6108))
- Update colorama requirement from <0.4.6,>=0.3.9 to >=0.3.9,<0.4.7 in /core ([#6144](https://github.com/dbt-labs/dbt-core/pull/6144))
- Bump mashumaro[msgpack] from 3.1.1 to 3.2 in /core ([#6375](https://github.com/dbt-labs/dbt-core/pull/6375))
- Update agate requirement from <1.6.4,>=1.6 to >=1.6,<1.7.1 in /core ([#6506](https://github.com/dbt-labs/dbt-core/pull/6506))
### Contributors
- [@NiallRees](https://github.com/NiallRees) ([#5859](https://github.com/dbt-labs/dbt-core/issues/5859))
- [@agpapa](https://github.com/agpapa) ([#6365](https://github.com/dbt-labs/dbt-core/issues/6365))
- [@andy-clapson](https://github.com/andy-clapson) ([dbt-docs/#5791](https://github.com/dbt-labs/dbt-docs/issues/5791))
- [@callum-mcdata](https://github.com/callum-mcdata) ([#6398](https://github.com/dbt-labs/dbt-core/issues/6398), [#6507](https://github.com/dbt-labs/dbt-core/issues/6507))
- [@chamini2](https://github.com/chamini2) ([#6041](https://github.com/dbt-labs/dbt-core/issues/6041))
- [@daniel-murray](https://github.com/daniel-murray) ([#5016](https://github.com/dbt-labs/dbt-core/issues/5016))
- [@dave-connors-3](https://github.com/dave-connors-3) ([#5680](https://github.com/dbt-labs/dbt-core/issues/5680), [#5990](https://github.com/dbt-labs/dbt-core/issues/5990), [#6625](https://github.com/dbt-labs/dbt-core/issues/6625))
- [@dbeatty10](https://github.com/dbeatty10) ([#6411](https://github.com/dbt-labs/dbt-core/issues/6411), [dbt-docs/#6368](https://github.com/dbt-labs/dbt-docs/issues/6368), [#6394](https://github.com/dbt-labs/dbt-core/issues/6394), [#6294](https://github.com/dbt-labs/dbt-core/issues/6294), [#6566](https://github.com/dbt-labs/dbt-core/issues/6566))
- [@devmessias](https://github.com/devmessias) ([#6309](https://github.com/dbt-labs/dbt-core/issues/6309))
- [@eltociear](https://github.com/eltociear) ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904))
- [@eve-johns](https://github.com/eve-johns) ([#6068](https://github.com/dbt-labs/dbt-core/issues/6068))
- [@haritamar](https://github.com/haritamar) ([#6246](https://github.com/dbt-labs/dbt-core/issues/6246))
- [@jared-rimmer](https://github.com/jared-rimmer) ([#5486](https://github.com/dbt-labs/dbt-core/issues/5486))
- [@josephberni](https://github.com/josephberni) ([#5016](https://github.com/dbt-labs/dbt-core/issues/5016))
- [@joshuataylor](https://github.com/joshuataylor) ([#6147](https://github.com/dbt-labs/dbt-core/issues/6147))
- [@justbldwn](https://github.com/justbldwn) ([#6241](https://github.com/dbt-labs/dbt-core/issues/6241), [#6245](https://github.com/dbt-labs/dbt-core/issues/6245), [#6269](https://github.com/dbt-labs/dbt-core/issues/6269))
- [@luke-bassett](https://github.com/luke-bassett) ([#1350](https://github.com/dbt-labs/dbt-core/issues/1350))
- [@max-sixty](https://github.com/max-sixty) ([#5946](https://github.com/dbt-labs/dbt-core/issues/5946), [#5983](https://github.com/dbt-labs/dbt-core/issues/5983), [#6028](https://github.com/dbt-labs/dbt-core/issues/6028))
- [@mivanicova](https://github.com/mivanicova) ([#6488](https://github.com/dbt-labs/dbt-core/issues/6488))
- [@nshuman1](https://github.com/nshuman1) ([dbt-docs/#6529](https://github.com/dbt-labs/dbt-docs/issues/6529))
- [@paulbenschmidt](https://github.com/paulbenschmidt) ([dbt-docs/#5880](https://github.com/dbt-labs/dbt-docs/issues/5880))
- [@pgoslatara](https://github.com/pgoslatara) ([#5929](https://github.com/dbt-labs/dbt-core/issues/5929))
- [@racheldaniel](https://github.com/racheldaniel) ([#6201](https://github.com/dbt-labs/dbt-core/issues/6201))
- [@timle2](https://github.com/timle2) ([#4205](https://github.com/dbt-labs/dbt-core/issues/4205))
- [@tmastny](https://github.com/tmastny) ([#6216](https://github.com/dbt-labs/dbt-core/issues/6216))

View File

@@ -1,8 +0,0 @@
## dbt-core 1.4.1 - January 26, 2023
### Fixes
- [Regression] exposure_content referenced incorrectly ([#6738](https://github.com/dbt-labs/dbt-core/issues/6738))
### Contributors
- [@Mathyoub](https://github.com/Mathyoub) ([#6738](https://github.com/dbt-labs/dbt-core/issues/6738))

View File

@@ -1,20 +0,0 @@
## dbt-core 1.4.2 - February 23, 2023
### Fixes
- Sort cli vars before hashing for partial parsing ([#6710](https://github.com/dbt-labs/dbt-core/issues/6710))
- Remove pin on packaging and stop using it for prerelease comparisons ([#6834](https://github.com/dbt-labs/dbt-core/issues/6834))
- Readd depends_on.macros to SeedNode, to support seeds with hooks calling macros ([#6806](https://github.com/dbt-labs/dbt-core/issues/6806))
- Fix regression of --quiet cli parameter behavior ([#6749](https://github.com/dbt-labs/dbt-core/issues/6749))
- Ensure results from hooks contain nodes when processing them ([#6796](https://github.com/dbt-labs/dbt-core/issues/6796))
- Always flush stdout after logging ([#6901](https://github.com/dbt-labs/dbt-core/issues/6901))
- Set relation_name in test nodes at compile time ([#6930](https://github.com/dbt-labs/dbt-core/issues/6930))
- Fix disabled definition in WritableManifest ([#6752](https://github.com/dbt-labs/dbt-core/issues/6752))
- Fix regression in logbook log output ([#7028](https://github.com/dbt-labs/dbt-core/issues/7028))
### Docs
- Fix JSON path to overview docs ([dbt-docs/#366](https://github.com/dbt-labs/dbt-docs/issues/366))
### Contributors
- [@halvorlu](https://github.com/halvorlu) ([#366](https://github.com/dbt-labs/dbt-core/issues/366))

View File

@@ -0,0 +1,7 @@
kind: Breaking Changes
body: Renaming Metric Spec Attributes
time: 2022-09-06T15:45:21.2769-05:00
custom:
Author: callum-mcdata
Issue: "5774"
PR: "5775"

View File

@@ -0,0 +1,7 @@
kind: "Dependency"
body: "Bump black from 22.6.0 to 22.8.0"
time: 2022-09-06T13:48:58.00000Z
custom:
Author: dependabot[bot]
Issue: 4904
PR: 5750

View File

@@ -0,0 +1,7 @@
kind: "Dependency"
body: "Bump python from 3.10.6-slim-bullseye to 3.10.7-slim-bullseye in /docker"
time: 2022-09-12T00:22:53.00000Z
custom:
Author: dependabot[bot]
Issue: 4904
PR: 5805

View File

@@ -0,0 +1,7 @@
kind: Docs
body: Refer to exposures by their label by default.
time: 2022-09-20T15:20:40.652948-05:00
custom:
Author: emmyoop
Issue: "306"
PR: "307"

View File

@@ -0,0 +1,7 @@
kind: Features
body: merge_exclude_columns for incremental materialization
time: 2022-07-16T14:21:16.592519-05:00
custom:
Author: dave-connors-3
Issue: "5260"
PR: "5457"

View File

@@ -0,0 +1,7 @@
kind: Features
body: Search current working directory for `profiles.yml`
time: 2022-08-25T19:50:23.940417-06:00
custom:
Author: dbeatty10
Issue: "5411"
PR: "5717"

View File

@@ -0,0 +1,7 @@
kind: Features
body: Adding the `window` parameter to the metric spec.
time: 2022-08-31T12:13:19.48356-05:00
custom:
Author: callum-mcdata
Issue: "5721"
PR: "5722"

View File

@@ -0,0 +1,7 @@
kind: Features
body: Add invocation args dict to ProviderContext class
time: 2022-09-08T08:13:15.17337+01:00
custom:
Author: jared-rimmer
Issue: "5524"
PR: "5782"

View File

@@ -0,0 +1,7 @@
kind: Features
body: Adds new cli framework
time: 2022-09-08T10:41:49.375734-05:00
custom:
Author: iknox-fa
Issue: "5526"
PR: "5647"

View File

@@ -0,0 +1,7 @@
kind: Features
body: Flags work with new Click CLI
time: 2022-09-08T12:36:50.386978-05:00
custom:
Author: iknox-fa
Issue: "5529"
PR: "5790"

View File

@@ -0,0 +1,7 @@
kind: Features
body: Add metadata env method to ProviderContext class
time: 2022-09-09T20:46:43.889302+01:00
custom:
Author: jared-rimmer
Issue: "5522"
PR: "5794"

View File

@@ -0,0 +1,7 @@
kind: Features
body: Array macros
time: 2022-09-12T22:22:27.475515-06:00
custom:
Author: graciegoheen dbeatty10
Issue: "5520"
PR: "5823"

View File

@@ -0,0 +1,7 @@
kind: Features
body: Add enabled config to exposures and metrics
time: 2022-09-13T09:59:24.445918-05:00
custom:
Author: emmyoop
Issue: "5422"
PR: "5815"

View File

@@ -0,0 +1,7 @@
kind: Features
body: add -fr flag shorthand
time: 2022-09-19T11:29:03.774678-05:00
custom:
Author: dave-connors-3
Issue: "5878"
PR: "5879"

View File

@@ -0,0 +1,7 @@
kind: Features
body: add type_boolean as a data type macro
time: 2022-09-19T23:14:14.9871+01:00
custom:
Author: jpmmcneill
Issue: "5739"
PR: "5875"

View File

@@ -0,0 +1,7 @@
kind: Features
body: Support .dbtignore in project root to ignore certain files being read by dbt
time: 2022-09-21T14:52:22.131627-07:00
custom:
Author: ChenyuLInx
Issue: "5733"
PR: "5897"

View File

@@ -0,0 +1,9 @@
kind: Features
body: This conditionally no-ops warehouse connection at compile depending on an env
var, disabling introspection/queries during compilation only. This is a temporary
solution to more complex permissions requirements for the semantic layer.
time: 2022-09-26T13:06:27.591061-05:00
custom:
Author: racheldaniel
Issue: "5936"
PR: "5926"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: Fix typos of comments in core/dbt/adapters/
time: 2022-08-22T19:42:38.593923+09:00
custom:
Author: yoiki
Issue: "5690"
PR: "5693"

View File

@@ -0,0 +1,8 @@
kind: Fixes
body: Include py.typed in MANIFEST.in. This enables packages that install dbt-core
from pypi to use mypy.
time: 2022-08-23T11:26:33.8415455-07:00
custom:
Author: panasenco
Issue: "5703"
PR: "5703"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: Removal of all .coverage files when using make clean command
time: 2022-09-03T15:54:09.741554082+01:00
custom:
Author: jared-rimmer
Issue: "5633"
PR: "5759"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: Remove temp files generated by unit tests
time: 2022-09-09T09:28:32.590208+02:00
custom:
Author: bbroeksema
Issue: "5631"
PR: "5749"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: Fix warnings as errors during tests
time: 2022-09-09T09:56:27.90654-05:00
custom:
Author: stu-k
Issue: "5424"
PR: "5800"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: Prevent event_history from holding references
time: 2022-09-16T09:17:23.273847-04:00
custom:
Author: gshank
Issue: "5848"
PR: "5858"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: ConfigSelectorMethod should check for bools
time: 2022-09-20T18:18:56.630628+01:00
custom:
Author: danielcmessias
Issue: "5890"
PR: "5889"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: shorthand for full refresh should be one character
time: 2022-09-22T08:39:26.948671-05:00
custom:
Author: dave-connors-3
Issue: "5878"
PR: "5908"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: Fix macro resolution order during static analysis for custom generic tests
time: 2022-09-23T14:32:26.857376+02:00
custom:
Author: jtcohen6
Issue: "5720"
PR: "5907"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: Fix race condition when invoking dbt via lib.py concurrently
time: 2022-09-23T17:45:04.405026-04:00
custom:
Author: drewbanin
Issue: "5919"
PR: "5921"

View File

@@ -0,0 +1,7 @@
kind: Under the Hood
body: Migrate integration test 014 but also fix the snapshot hard delete test's timezone logic and force all integration tests to run flags.set_from_args to force environment variables are accessible to all integration test threads.
time: 2022-09-05T00:17:49.564534-07:00
custom:
Author: versusfacit
Issue: "5760"
PR: "5760"

View File

@@ -0,0 +1,7 @@
kind: Under the Hood
body: Support dbt-metrics compilation by rebuilding flat_graph
time: 2022-09-08T14:56:44.173322-04:00
custom:
Author: gshank
Issue: "5525"
PR: "5786"

View File

@@ -0,0 +1,8 @@
kind: Under the Hood
body: Reworking the way we define the window attribute of metrics to match freshness
tests
time: 2022-09-08T18:07:31.532608-05:00
custom:
Author: callum-mcdata
Issue: "5722"
PR: "5793"

View File

@@ -0,0 +1,7 @@
kind: Under the Hood
body: Add PythonJobHelper base class in core and add more type checking
time: 2022-09-09T11:52:20.419364-07:00
custom:
Author: ChenyuLInx
Issue: "5802"
PR: "5802"

View File

@@ -0,0 +1,7 @@
kind: Under the Hood
body: Convert default selector tests to pytest
time: 2022-09-12T13:40:00.625912-05:00
custom:
Author: stu-k
Issue: "5728"
PR: "5820"

View File

@@ -0,0 +1,7 @@
kind: Under the Hood
body: The link did not go to the anchor directly, now it does
time: 2022-09-12T14:00:35.899828+02:00
custom:
Author: sdebruyn
Issue: "5813"
PR: "5814"

View File

@@ -0,0 +1,7 @@
kind: Under the Hood
body: remove key as reserved keyword from test_bool_or
time: 2022-09-12T19:03:41.481601+02:00
custom:
Author: sdebruyn
Issue: "5817"
PR: "5818"

View File

@@ -0,0 +1,7 @@
kind: Under the Hood
body: Compatibiltiy for metric attribute renaming
time: 2022-09-13T11:17:44.953536+02:00
custom:
Author: jtcohen6 callum-mcdata
Issue: "5807"
PR: "5825"

View File

@@ -0,0 +1,7 @@
kind: Under the Hood
body: Add name validation for metrics
time: 2022-09-14T13:26:32.387524-05:00
custom:
Author: emmyoop
Issue: "5456"
PR: "5841"

View File

@@ -0,0 +1,7 @@
kind: Under the Hood
body: Validate exposure name and add label
time: 2022-09-14T15:00:58.982982-05:00
custom:
Author: emmyoop
Issue: "5606"
PR: "5844"

View File

@@ -0,0 +1,7 @@
kind: Under the Hood
body: remove source quoting setting in adapter tests
time: 2022-09-14T19:39:33.688385+02:00
custom:
Author: sdebruyn
Issue: "5836"
PR: "5839"

View File

@@ -0,0 +1,7 @@
kind: Under the Hood
body: Adding validation for metric expression attribute
time: 2022-09-16T15:47:12.799002-05:00
custom:
Author: callum-mcdata
Issue: "5871"
PR: "5873"

View File

@@ -0,0 +1,7 @@
kind: Under the Hood
body: Profiling and Adapter Management work with Click CLI
time: 2022-09-20T14:48:42.070256-05:00
custom:
Author: iknox-fa
Issue: "5531"
PR: "5892"

View File

@@ -0,0 +1,7 @@
kind: Under the Hood
body: Reparse references to deleted metric
time: 2022-09-23T13:35:25.681656-04:00
custom:
Author: gshank
Issue: "5444"
PR: "5920"

113
.changie.yaml Normal file → Executable file
View File

@@ -6,67 +6,19 @@ changelogPath: CHANGELOG.md
versionExt: md
versionFormat: '## dbt-core {{.Version}} - {{.Time.Format "January 02, 2006"}}'
kindFormat: '### {{.Kind}}'
changeFormat: |-
{{- $IssueList := list }}
{{- $changes := splitList " " $.Custom.Issue }}
{{- range $issueNbr := $changes }}
{{- $changeLink := "[#nbr](https://github.com/dbt-labs/dbt-core/issues/nbr)" | replace "nbr" $issueNbr }}
{{- $IssueList = append $IssueList $changeLink }}
{{- end -}}
- {{.Body}} ({{ range $index, $element := $IssueList }}{{if $index}}, {{end}}{{$element}}{{end}})
changeFormat: '- {{.Body}} ([#{{.Custom.Issue}}](https://github.com/dbt-labs/dbt-core/issues/{{.Custom.Issue}}), [#{{.Custom.PR}}](https://github.com/dbt-labs/dbt-core/pull/{{.Custom.PR}}))'
kinds:
- label: Breaking Changes
- label: Features
- label: Fixes
- label: Docs
changeFormat: |-
{{- $IssueList := list }}
{{- $changes := splitList " " $.Custom.Issue }}
{{- range $issueNbr := $changes }}
{{- $changeLink := "[dbt-docs/#nbr](https://github.com/dbt-labs/dbt-docs/issues/nbr)" | replace "nbr" $issueNbr }}
{{- $IssueList = append $IssueList $changeLink }}
{{- end -}}
- {{.Body}} ({{ range $index, $element := $IssueList }}{{if $index}}, {{end}}{{$element}}{{end}})
changeFormat: '- {{.Body}} ([dbt-docs/#{{.Custom.Issue}}](https://github.com/dbt-labs/dbt-docs/issues/{{.Custom.Issue}}), [dbt-docs/#{{.Custom.PR}}](https://github.com/dbt-labs/dbt-docs/pull/{{.Custom.PR}}))'
- label: Under the Hood
- label: Dependencies
changeFormat: |-
{{- $PRList := list }}
{{- $changes := splitList " " $.Custom.PR }}
{{- range $pullrequest := $changes }}
{{- $changeLink := "[#nbr](https://github.com/dbt-labs/dbt-core/pull/nbr)" | replace "nbr" $pullrequest }}
{{- $PRList = append $PRList $changeLink }}
{{- end -}}
- {{.Body}} ({{ range $index, $element := $PRList }}{{if $index}}, {{end}}{{$element}}{{end}})
skipGlobalChoices: true
additionalChoices:
- key: Author
label: GitHub Username(s) (separated by a single space if multiple)
type: string
minLength: 3
- key: PR
label: GitHub Pull Request Number (separated by a single space if multiple)
type: string
minLength: 1
changeFormat: '- {{.Body}} ({{if ne .Custom.Issue ""}}[#{{.Custom.Issue}}](https://github.com/dbt-labs/dbt-core/issues/{{.Custom.Issue}}), {{end}}[#{{.Custom.PR}}](https://github.com/dbt-labs/dbt-core/pull/{{.Custom.PR}}))'
- label: Security
changeFormat: |-
{{- $PRList := list }}
{{- $changes := splitList " " $.Custom.PR }}
{{- range $pullrequest := $changes }}
{{- $changeLink := "[#nbr](https://github.com/dbt-labs/dbt-core/pull/nbr)" | replace "nbr" $pullrequest }}
{{- $PRList = append $PRList $changeLink }}
{{- end -}}
- {{.Body}} ({{ range $index, $element := $PRList }}{{if $index}}, {{end}}{{$element}}{{end}})
skipGlobalChoices: true
additionalChoices:
- key: Author
label: GitHub Username(s) (separated by a single space if multiple)
type: string
minLength: 3
- key: PR
label: GitHub Pull Request Number (separated by a single space if multiple)
type: string
minLength: 1
changeFormat: '- {{.Body}} ({{if ne .Custom.Issue ""}}[#{{.Custom.Issue}}](https://github.com/dbt-labs/dbt-core/issues/{{.Custom.Issue}}), {{end}}[#{{.Custom.PR}}](https://github.com/dbt-labs/dbt-core/pull/{{.Custom.PR}}))'
newlines:
afterChangelogHeader: 1
@@ -81,47 +33,42 @@ custom:
type: string
minLength: 3
- key: Issue
label: GitHub Issue Number (separated by a single space if multiple)
type: string
minLength: 1
label: GitHub Issue Number
type: int
minInt: 1
- key: PR
label: GitHub Pull Request Number
type: int
minInt: 1
footerFormat: |
{{- $contributorDict := dict }}
{{- /* any names added to this list should be all lowercase for later matching purposes */}}
{{- $core_team := list "michelleark" "peterallenwebb" "emmyoop" "nathaniel-may" "gshank" "leahwicz" "chenyulinx" "stu-k" "iknox-fa" "versusfacit" "mcknight-42" "jtcohen6" "aranke" "dependabot[bot]" "snyk-bot" "colin-rogers-dbt" }}
{{- $core_team := list "peterallenwebb" "emmyoop" "nathaniel-may" "gshank" "leahwicz" "chenyulinx" "stu-k" "iknox-fa" "versusfacit" "mcknight-42" "jtcohen6" "dependabot[bot]" "snyk-bot" }}
{{- range $change := .Changes }}
{{- $authorList := splitList " " $change.Custom.Author }}
{{- /* loop through all authors for a single changelog */}}
{{- /* loop through all authors for a PR */}}
{{- range $author := $authorList }}
{{- $authorLower := lower $author }}
{{- /* we only want to include non-core team contributors */}}
{{- if not (has $authorLower $core_team)}}
{{- $changeList := splitList " " $change.Custom.Author }}
{{- $IssueList := list }}
{{- $changeLink := $change.Kind }}
{{- if or (eq $change.Kind "Dependencies") (eq $change.Kind "Security") }}
{{- $changes := splitList " " $change.Custom.PR }}
{{- range $issueNbr := $changes }}
{{- $changeLink := "[#nbr](https://github.com/dbt-labs/dbt-core/pull/nbr)" | replace "nbr" $issueNbr }}
{{- $IssueList = append $IssueList $changeLink }}
{{- end -}}
{{- else }}
{{- $changes := splitList " " $change.Custom.Issue }}
{{- range $issueNbr := $changes }}
{{- $changeLink := "[#nbr](https://github.com/dbt-labs/dbt-core/issues/nbr)" | replace "nbr" $issueNbr }}
{{- $IssueList = append $IssueList $changeLink }}
{{- end -}}
{{- end }}
{{- /* check if this contributor has other changes associated with them already */}}
{{- if hasKey $contributorDict $author }}
{{- $contributionList := get $contributorDict $author }}
{{- $contributionList = concat $contributionList $IssueList }}
{{- $contributorDict := set $contributorDict $author $contributionList }}
{{- else }}
{{- $contributionList := $IssueList }}
{{- $contributorDict := set $contributorDict $author $contributionList }}
{{- end }}
{{- end}}
{{- /* Docs kind link back to dbt-docs instead of dbt-core PRs */}}
{{- $prLink := $change.Kind }}
{{- if eq $change.Kind "Docs" }}
{{- $prLink = "[dbt-docs/#pr](https://github.com/dbt-labs/dbt-docs/pull/pr)" | replace "pr" $change.Custom.PR }}
{{- else }}
{{- $prLink = "[#pr](https://github.com/dbt-labs/dbt-core/pull/pr)" | replace "pr" $change.Custom.PR }}
{{- end }}
{{- /* check if this contributor has other PRs associated with them already */}}
{{- if hasKey $contributorDict $author }}
{{- $prList := get $contributorDict $author }}
{{- $prList = append $prList $prLink }}
{{- $contributorDict := set $contributorDict $author $prList }}
{{- else }}
{{- $prList := list $prLink }}
{{- $contributorDict := set $contributorDict $author $prList }}
{{- end }}
{{- end}}
{{- end}}
{{- end }}
{{- /* no indentation here for formatting so the final markdown doesn't have unneeded indentations */}}

2
.gitattributes vendored
View File

@@ -1,2 +0,0 @@
core/dbt/include/index.html binary
tests/functional/artifacts/data/state/*/manifest.json binary

View File

@@ -40,7 +40,7 @@ jobs:
matrix:
include:
- label: "dependencies"
changie_kind: "Dependencies"
changie_kind: "Dependency"
- label: "snyk"
changie_kind: "Security"
runs-on: ubuntu-latest
@@ -58,4 +58,4 @@ jobs:
commit_message: "Add automated changelog yaml from template for bot PR"
changie_kind: ${{ matrix.changie_kind }}
label: ${{ matrix.label }}
custom_changelog_string: "custom:\n Author: ${{ github.event.pull_request.user.login }}\n PR: ${{ github.event.pull_request.number }}"
custom_changelog_string: "custom:\n Author: ${{ github.event.pull_request.user.login }}\n Issue: 4904\n PR: ${{ github.event.pull_request.number }}"

View File

@@ -1,165 +0,0 @@
# **what?**
# On push, if anything in core/dbt/docs or core/dbt/cli has been
# created or modified, regenerate the CLI API docs using sphinx.
# **why?**
# We watch for changes in core/dbt/cli because the CLI API docs rely on click
# and all supporting flags/params to be generated. We watch for changes in
# core/dbt/docs since any changes to sphinx configuration or any of the
# .rst files there could result in a differently build final index.html file.
# **when?**
# Whenever a change has been pushed to a branch, and only if there is a diff
# between the PR branch and main's core/dbt/cli and or core/dbt/docs dirs.
# TODO: add bot comment to PR informing contributor that the docs have been committed
# TODO: figure out why github action triggered pushes cause github to fail to report
# the status of jobs
name: Generate CLI API docs
on:
pull_request:
permissions:
contents: write
pull-requests: write
env:
CLI_DIR: ${{ github.workspace }}/core/dbt/cli
DOCS_DIR: ${{ github.workspace }}/core/dbt/docs
DOCS_BUILD_DIR: ${{ github.workspace }}/core/dbt/docs/build
jobs:
check_gen:
name: check if generation needed
runs-on: ubuntu-latest
if: ${{ github.event.pull_request.head.repo.fork == false }}
outputs:
cli_dir_changed: ${{ steps.check_cli.outputs.cli_dir_changed }}
docs_dir_changed: ${{ steps.check_docs.outputs.docs_dir_changed }}
steps:
- name: "[DEBUG] print variables"
run: |
echo "env.CLI_DIR: ${{ env.CLI_DIR }}"
echo "env.DOCS_BUILD_DIR: ${{ env.DOCS_BUILD_DIR }}"
echo "env.DOCS_DIR: ${{ env.DOCS_DIR }}"
- name: git checkout
uses: actions/checkout@v3
with:
fetch-depth: 0
ref: ${{ github.head_ref }}
- name: set shas
id: set_shas
run: |
THIS_SHA=$(git rev-parse @)
LAST_SHA=$(git rev-parse @~1)
echo "this sha: $THIS_SHA"
echo "last sha: $LAST_SHA"
echo "this_sha=$THIS_SHA" >> $GITHUB_OUTPUT
echo "last_sha=$LAST_SHA" >> $GITHUB_OUTPUT
- name: check for changes in core/dbt/cli
id: check_cli
run: |
CLI_DIR_CHANGES=$(git diff \
${{ steps.set_shas.outputs.last_sha }} \
${{ steps.set_shas.outputs.this_sha }} \
-- ${{ env.CLI_DIR }})
if [ -n "$CLI_DIR_CHANGES" ]; then
echo "changes found"
echo $CLI_DIR_CHANGES
echo "cli_dir_changed=true" >> $GITHUB_OUTPUT
exit 0
fi
echo "cli_dir_changed=false" >> $GITHUB_OUTPUT
echo "no changes found"
- name: check for changes in core/dbt/docs
id: check_docs
if: steps.check_cli.outputs.cli_dir_changed == 'false'
run: |
DOCS_DIR_CHANGES=$(git diff --name-only \
${{ steps.set_shas.outputs.last_sha }} \
${{ steps.set_shas.outputs.this_sha }} \
-- ${{ env.DOCS_DIR }} ':!${{ env.DOCS_BUILD_DIR }}')
DOCS_BUILD_DIR_CHANGES=$(git diff --name-only \
${{ steps.set_shas.outputs.last_sha }} \
${{ steps.set_shas.outputs.this_sha }} \
-- ${{ env.DOCS_BUILD_DIR }})
if [ -n "$DOCS_DIR_CHANGES" ] && [ -z "$DOCS_BUILD_DIR_CHANGES" ]; then
echo "changes found"
echo $DOCS_DIR_CHANGES
echo "docs_dir_changed=true" >> $GITHUB_OUTPUT
exit 0
fi
echo "docs_dir_changed=false" >> $GITHUB_OUTPUT
echo "no changes found"
gen_docs:
name: generate docs
runs-on: ubuntu-latest
needs: [check_gen]
if: |
needs.check_gen.outputs.cli_dir_changed == 'true'
|| needs.check_gen.outputs.docs_dir_changed == 'true'
steps:
- name: "[DEBUG] print variables"
run: |
echo "env.DOCS_DIR: ${{ env.DOCS_DIR }}"
echo "github head_ref: ${{ github.head_ref }}"
- name: git checkout
uses: actions/checkout@v3
with:
ref: ${{ github.head_ref }}
- name: install python
uses: actions/setup-python@v4.3.0
with:
python-version: 3.8
- name: install dev requirements
run: |
python3 -m venv env
source env/bin/activate
python -m pip install --upgrade pip
pip install -r requirements.txt -r dev-requirements.txt
- name: generate docs
run: |
source env/bin/activate
cd ${{ env.DOCS_DIR }}
echo "cleaning existing docs"
make clean
echo "creating docs"
make html
- name: debug
run: |
echo ">>>>> status"
git status
echo ">>>>> remotes"
git remote -v
echo ">>>>> branch"
git branch -v
echo ">>>>> log"
git log --pretty=oneline | head -5
- name: commit docs
run: |
git config user.name 'Github Build Bot'
git config user.email 'buildbot@fishtownanalytics.com'
git commit -am "Add generated CLI API docs"
git push -u origin ${{ github.head_ref }}

View File

@@ -45,9 +45,7 @@ jobs:
uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v4.3.0
with:
python-version: '3.8'
uses: actions/setup-python@v2
- name: Install python dependencies
run: |
@@ -73,7 +71,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
python-version: ["3.7", "3.8", "3.9", "3.10"]
env:
TOXENV: "unit"
@@ -84,7 +82,7 @@ jobs:
uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4.3.0
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
@@ -118,8 +116,8 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
os: [ubuntu-20.04]
python-version: ["3.7", "3.8", "3.9", "3.10"]
os: [ubuntu-latest]
include:
- python-version: 3.8
os: windows-latest
@@ -139,7 +137,7 @@ jobs:
uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4.3.0
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
@@ -192,9 +190,9 @@ jobs:
uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v4.3.0
uses: actions/setup-python@v2
with:
python-version: '3.8'
python-version: 3.8
- name: Install python dependencies
run: |

View File

@@ -1,109 +0,0 @@
# **what?**
# Nightly releases to GitHub and PyPI. This workflow produces the following outcome:
# - generate and validate data for night release (commit SHA, version number, release branch);
# - pass data to release workflow;
# - night release will be pushed to GitHub as a draft release;
# - night build will be pushed to test PyPI;
#
# **why?**
# Ensure an automated and tested release process for nightly builds
#
# **when?**
# This workflow runs on schedule or can be run manually on demand.
name: Nightly Test Release to GitHub and PyPI
on:
workflow_dispatch: # for manual triggering
schedule:
- cron: 0 9 * * *
permissions:
contents: write # this is the permission that allows creating a new release
defaults:
run:
shell: bash
env:
RELEASE_BRANCH: "main"
jobs:
aggregate-release-data:
runs-on: ubuntu-latest
outputs:
commit_sha: ${{ steps.resolve-commit-sha.outputs.release_commit }}
version_number: ${{ steps.nightly-release-version.outputs.number }}
release_branch: ${{ steps.release-branch.outputs.name }}
steps:
- name: "Checkout ${{ github.repository }} Branch ${{ env.RELEASE_BRANCH }}"
uses: actions/checkout@v3
with:
ref: ${{ env.RELEASE_BRANCH }}
- name: "Resolve Commit To Release"
id: resolve-commit-sha
run: |
commit_sha=$(git rev-parse HEAD)
echo "release_commit=$commit_sha" >> $GITHUB_OUTPUT
- name: "Get Current Version Number"
id: version-number-sources
run: |
current_version=`awk -F"current_version = " '{print $2}' .bumpversion.cfg | tr '\n' ' '`
echo "current_version=$current_version" >> $GITHUB_OUTPUT
- name: "Audit Version And Parse Into Parts"
id: semver
uses: dbt-labs/actions/parse-semver@v1.1.0
with:
version: ${{ steps.version-number-sources.outputs.current_version }}
- name: "Get Current Date"
id: current-date
run: echo "date=$(date +'%m%d%Y')" >> $GITHUB_OUTPUT
- name: "Generate Nightly Release Version Number"
id: nightly-release-version
run: |
number="${{ steps.semver.outputs.version }}.dev${{ steps.current-date.outputs.date }}+nightly"
echo "number=$number" >> $GITHUB_OUTPUT
- name: "Audit Nightly Release Version And Parse Into Parts"
uses: dbt-labs/actions/parse-semver@v1.1.0
with:
version: ${{ steps.nightly-release-version.outputs.number }}
- name: "Set Release Branch"
id: release-branch
run: |
echo "name=${{ env.RELEASE_BRANCH }}" >> $GITHUB_OUTPUT
log-outputs-aggregate-release-data:
runs-on: ubuntu-latest
needs: [aggregate-release-data]
steps:
- name: "[DEBUG] Log Outputs"
run: |
echo commit_sha : ${{ needs.aggregate-release-data.outputs.commit_sha }}
echo version_number: ${{ needs.aggregate-release-data.outputs.version_number }}
echo release_branch: ${{ needs.aggregate-release-data.outputs.release_branch }}
release-github-pypi:
needs: [aggregate-release-data]
uses: ./.github/workflows/release.yml
with:
sha: ${{ needs.aggregate-release-data.outputs.commit_sha }}
target_branch: ${{ needs.aggregate-release-data.outputs.release-branch }}
version_number: ${{ needs.aggregate-release-data.outputs.version_number }}
build_script_path: "scripts/build-dist.sh"
env_setup_script_path: "scripts/env-setup.sh"
s3_bucket_name: "core-team-artifacts"
package_test_command: "dbt --version"
test_run: true
nightly_release: true
secrets: inherit

View File

@@ -39,7 +39,7 @@ jobs:
max-parallel: 1
fail-fast: false
matrix:
branch: [1.0.latest, 1.1.latest, 1.2.latest, 1.3.latest, main]
branch: [1.0.latest, 1.1.latest, 1.2.latest, main]
steps:
- name: Call CI workflow for ${{ matrix.branch }} branch

View File

@@ -1,110 +1,24 @@
# **what?**
# Release workflow provides the following steps:
# - checkout the given commit;
# - validate version in sources and changelog file for given version;
# - bump the version and generate a changelog if needed;
# - merge all changes to the target branch if needed;
# - run unit and integration tests against given commit;
# - build and package that SHA;
# - release it to GitHub and PyPI with that specific build;
#
# Take the given commit, run unit tests specifically on that sha, build and
# package it, and then release to GitHub and PyPi with that specific build
# **why?**
# Ensure an automated and tested release process
#
# **when?**
# This workflow can be run manually on demand or can be called by other workflows
name: Release to GitHub and PyPI
# **when?**
# This will only run manually with a given sha and version
name: Release to GitHub and PyPi
on:
workflow_dispatch:
inputs:
sha:
description: "The last commit sha in the release"
type: string
required: true
target_branch:
description: "The branch to release from"
type: string
required: true
description: 'The last commit sha in the release'
required: true
version_number:
description: "The release version number (i.e. 1.0.0b1)"
type: string
required: true
build_script_path:
description: "Build script path"
type: string
default: "scripts/build-dist.sh"
required: true
env_setup_script_path:
description: "Environment setup script path"
type: string
default: "scripts/env-setup.sh"
required: false
s3_bucket_name:
description: "AWS S3 bucket name"
type: string
default: "core-team-artifacts"
required: true
package_test_command:
description: "Package test command"
type: string
default: "dbt --version"
required: true
test_run:
description: "Test run (Publish release as draft)"
type: boolean
default: true
required: false
nightly_release:
description: "Nightly release to dev environment"
type: boolean
default: false
required: false
workflow_call:
inputs:
sha:
description: "The last commit sha in the release"
type: string
required: true
target_branch:
description: "The branch to release from"
type: string
required: true
version_number:
description: "The release version number (i.e. 1.0.0b1)"
type: string
required: true
build_script_path:
description: "Build script path"
type: string
default: "scripts/build-dist.sh"
required: true
env_setup_script_path:
description: "Environment setup script path"
type: string
default: "scripts/env-setup.sh"
required: false
s3_bucket_name:
description: "AWS S3 bucket name"
type: string
default: "core-team-artifacts"
required: true
package_test_command:
description: "Package test command"
type: string
default: "dbt --version"
required: true
test_run:
description: "Test run (Publish release as draft)"
type: boolean
default: true
required: false
nightly_release:
description: "Nightly release to dev environment"
type: boolean
default: false
required: false
description: 'The release version number (i.e. 1.0.0b1)'
required: true
permissions:
contents: write # this is the permission that allows creating a new release
@@ -114,117 +28,175 @@ defaults:
shell: bash
jobs:
log-inputs:
name: Log Inputs
unit:
name: Unit test
runs-on: ubuntu-latest
env:
TOXENV: "unit"
steps:
- name: "[DEBUG] Print Variables"
- name: Check out the repository
uses: actions/checkout@v2
with:
persist-credentials: false
ref: ${{ github.event.inputs.sha }}
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: 3.8
- name: Install python dependencies
run: |
echo The last commit sha in the release: ${{ inputs.sha }}
echo The branch to release from: ${{ inputs.target_branch }}
echo The release version number: ${{ inputs.version_number }}
echo Build script path: ${{ inputs.build_script_path }}
echo Environment setup script path: ${{ inputs.env_setup_script_path }}
echo AWS S3 bucket name: ${{ inputs.s3_bucket_name }}
echo Package test command: ${{ inputs.package_test_command }}
echo Test run: ${{ inputs.test_run }}
echo Nightly release: ${{ inputs.nightly_release }}
pip install --user --upgrade pip
pip install tox
pip --version
tox --version
bump-version-generate-changelog:
name: Bump package version, Generate changelog
- name: Run tox
run: tox
uses: dbt-labs/dbt-release/.github/workflows/release-prep.yml@main
with:
sha: ${{ inputs.sha }}
version_number: ${{ inputs.version_number }}
target_branch: ${{ inputs.target_branch }}
env_setup_script_path: ${{ inputs.env_setup_script_path }}
test_run: ${{ inputs.test_run }}
nightly_release: ${{ inputs.nightly_release }}
secrets:
FISHTOWN_BOT_PAT: ${{ secrets.FISHTOWN_BOT_PAT }}
log-outputs-bump-version-generate-changelog:
name: "[Log output] Bump package version, Generate changelog"
if: ${{ !failure() && !cancelled() }}
needs: [bump-version-generate-changelog]
build:
name: build packages
runs-on: ubuntu-latest
steps:
- name: Print variables
- name: Check out the repository
uses: actions/checkout@v2
with:
persist-credentials: false
ref: ${{ github.event.inputs.sha }}
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: 3.8
- name: Install python dependencies
run: |
echo Final SHA : ${{ needs.bump-version-generate-changelog.outputs.final_sha }}
echo Changelog path: ${{ needs.bump-version-generate-changelog.outputs.changelog_path }}
pip install --user --upgrade pip
pip install --upgrade setuptools wheel twine check-wheel-contents
pip --version
build-test-package:
name: Build, Test, Package
if: ${{ !failure() && !cancelled() }}
needs: [bump-version-generate-changelog]
- name: Build distributions
run: ./scripts/build-dist.sh
uses: dbt-labs/dbt-release/.github/workflows/build.yml@main
- name: Show distributions
run: ls -lh dist/
with:
sha: ${{ needs.bump-version-generate-changelog.outputs.final_sha }}
version_number: ${{ inputs.version_number }}
changelog_path: ${{ needs.bump-version-generate-changelog.outputs.changelog_path }}
build_script_path: ${{ inputs.build_script_path }}
s3_bucket_name: ${{ inputs.s3_bucket_name }}
package_test_command: ${{ inputs.package_test_command }}
test_run: ${{ inputs.test_run }}
nightly_release: ${{ inputs.nightly_release }}
- name: Check distribution descriptions
run: |
twine check dist/*
secrets:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
- name: Check wheel contents
run: |
check-wheel-contents dist/*.whl --ignore W007,W008
- uses: actions/upload-artifact@v2
with:
name: dist
path: |
dist/
!dist/dbt-${{github.event.inputs.version_number}}.tar.gz
test-build:
name: verify packages
needs: [build, unit]
runs-on: ubuntu-latest
steps:
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: 3.8
- name: Install python dependencies
run: |
pip install --user --upgrade pip
pip install --upgrade wheel
pip --version
- uses: actions/download-artifact@v2
with:
name: dist
path: dist/
- name: Show distributions
run: ls -lh dist/
- name: Install wheel distributions
run: |
find ./dist/*.whl -maxdepth 1 -type f | xargs pip install --force-reinstall --find-links=dist/
- name: Check wheel distributions
run: |
dbt --version
- name: Install source distributions
run: |
find ./dist/*.gz -maxdepth 1 -type f | xargs pip install --force-reinstall --find-links=dist/
- name: Check source distributions
run: |
dbt --version
github-release:
name: GitHub Release
if: ${{ !failure() && !cancelled() }}
needs: [bump-version-generate-changelog, build-test-package]
needs: test-build
uses: dbt-labs/dbt-release/.github/workflows/github-release.yml@main
runs-on: ubuntu-latest
with:
sha: ${{ needs.bump-version-generate-changelog.outputs.final_sha }}
version_number: ${{ inputs.version_number }}
changelog_path: ${{ needs.bump-version-generate-changelog.outputs.changelog_path }}
test_run: ${{ inputs.test_run }}
steps:
- uses: actions/download-artifact@v2
with:
name: dist
path: '.'
# Need to set an output variable because env variables can't be taken as input
# This is needed for the next step with releasing to GitHub
- name: Find release type
id: release_type
env:
IS_PRERELEASE: ${{ contains(github.event.inputs.version_number, 'rc') || contains(github.event.inputs.version_number, 'b') }}
run: |
echo ::set-output name=isPrerelease::$IS_PRERELEASE
- name: Creating GitHub Release
uses: softprops/action-gh-release@v1
with:
name: dbt-core v${{github.event.inputs.version_number}}
tag_name: v${{github.event.inputs.version_number}}
prerelease: ${{ steps.release_type.outputs.isPrerelease }}
target_commitish: ${{github.event.inputs.sha}}
body: |
[Release notes](https://github.com/dbt-labs/dbt-core/blob/main/CHANGELOG.md)
files: |
dbt_postgres-${{github.event.inputs.version_number}}-py3-none-any.whl
dbt_core-${{github.event.inputs.version_number}}-py3-none-any.whl
dbt-postgres-${{github.event.inputs.version_number}}.tar.gz
dbt-core-${{github.event.inputs.version_number}}.tar.gz
pypi-release:
name: PyPI Release
name: Pypi release
needs: [github-release]
runs-on: ubuntu-latest
uses: dbt-labs/dbt-release/.github/workflows/pypi-release.yml@main
needs: github-release
with:
version_number: ${{ inputs.version_number }}
test_run: ${{ inputs.test_run }}
environment: PypiProd
steps:
- uses: actions/download-artifact@v2
with:
name: dist
path: 'dist'
secrets:
PYPI_API_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
TEST_PYPI_API_TOKEN: ${{ secrets.TEST_PYPI_API_TOKEN }}
slack-notification:
name: Slack Notification
if: ${{ failure() && (!inputs.test_run || inputs.nightly_release) }}
needs:
[
bump-version-generate-changelog,
build-test-package,
github-release,
pypi-release,
]
uses: dbt-labs/dbt-release/.github/workflows/slack-post-notification.yml@main
with:
status: "failure"
secrets:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_DEV_CORE_ALERTS }}
- name: Publish distribution to PyPI
uses: pypa/gh-action-pypi-publish@v1.4.2
with:
password: ${{ secrets.PYPI_API_TOKEN }}

Some files were not shown because too many files have changed in this diff Show More