Compare commits

...

267 Commits

Author SHA1 Message Date
Gerda Shank
e983ee4a75 Update sqslparse to 0.4.4 2023-06-22 17:37:23 -04:00
FishtownBuildBot
533988233e [Automated] Merged prep-release/1.6.0b5_5349512955 into target main during release process 2023-06-22 15:07:48 -05:00
Peter Webb
8bc0e77a1d CT-2719: Rename the semantic_nodes collection on the manifest to semantic_models (#7927) 2023-06-22 15:59:54 -04:00
Github Build Bot
1c93c9bb58 Bumping version to 1.6.0b5 and generate changelog 2023-06-22 19:27:29 +00:00
Michelle Ark
6d7b32977c Fix: safe remove of external nodes from nodes.depends_on (#7923) 2023-06-22 13:42:06 -04:00
Michelle Ark
bf15466bec UninstalledPackagesFoundError references correct packages specified path (#7886) 2023-06-22 11:40:59 -04:00
Gerda Shank
fb1ebe48f0 Resolve SemanticModel refs in the same way as other refs (#7895) 2023-06-22 11:26:08 -04:00
Peter Webb
de65697ff9 Further Integrate Semantic Models (#7917)
* CT-2651: Add Semantic Models to the manifest and various pieces of graph linking code

* CT-2651: Finish integrating semantic models into the partial parsing system

* CT-2651: More semantic model details for partial parsing

* CT-2651: Remove merged references to project_dependencies

* CT-2651: Revise changelog entry

* CT-2651: Disable unit test until partial parsing of semantic models is complete.

* CT-2651: Temporarily disable an apparently-flaky test.
2023-06-21 18:31:54 -04:00
Michelle Ark
ecf90d689e Refactor/unify public and model nodes (#7891) 2023-06-21 17:12:15 -04:00
Mila Page
4cdeff11cd Remove --config-dir instead of fixing #7774 (#7793)
* Stringfy the dir always to solve the bug.

* Add changelog

---------

Co-authored-by: Mila Page <versusfacit@users.noreply.github.com>
2023-06-21 11:36:47 -07:00
Gerda Shank
9ff2f6e430 Do not jinja render packages from dependencies yml (#7910)
* Add some comments to methods constructing Project/RuntimeConfig

* Save flag that packages dict came from dependencies.yml

* Test for not rendering packages_dict

* Changie

* Ensure packages_yml_dict and dependencies_yml_dict are dictionaries

* Ensure "packages" passed to render_packages is a dict
2023-06-21 14:33:23 -04:00
Jeremy Cohen
73a0dc6d14 Reorganize, annotate, and revise dependency pins (#7368)
* Reorganize + annotate dependencies

* Loosen pins on other dbt-labs packages

* Minor pins, no patch pins

* Rm support for py37

* Fix protobuf pin

* Bump networkx upper bound #6551
2023-06-21 07:08:43 +02:00
Kshitij Aranke
0a1c73e0fd Fixes #7753: Fix regression in run-operation to not require the name of the package to run (#7811) 2023-06-20 11:19:16 -05:00
Quigley Malcolm
8653ffc5a4 Upgrade core to support dbt-semantic-interfaces 0.1.0dev7 (#7903)
* Bump DSI dependency version to 0.1.0dev7

* Cleaner DSI type enum importing

Previoulsy we had to use individual import paths for each type enum
that dbt-semantic-interfaces provided. However, dbt-semantic-interfaces
has been updated to allow for importing all the type enums from a
singular path.

* Cleaner DSI protocol importing

Previoulsy we had to use individual import paths for each protocol
that dbt-semantic-interfaces provided. However, dbt-semantic-interfaces
has been updated to allow for importing all the protocols from a
singular path.

* Add semantic protocol satisifcation test for metric type params

* Replace `metric.type_params.measures` with `metric.type_params.input_measures`

In DSI 0.1.0dev7 `measures` on metric type params became `input_measures`.
Additionally `input_measures` should not be user specified but something
we compile at parse time, thus we've removed it from `UnparsedMetricTypeParams`.
Finally, actually populating `input_measures` is somewhat complicated due
to the existance of derived metrics, thus that work is being pushed
off to CT-2707.

* Update metric numerator/denominator to be `MetricInput`s

In DSI 0.1.0dev7 `metric.type_params.numerator` and `metric.type_params.denominator`
switched from being `MetricInputMeasure`s to `MetricInput`s. This
commit reflects that change. Additionally, some helper functions on
metric type params were removed related to the numerator and denominator.
Thus we've removed them respectively in this commit.

* Add protocol satisfaction tests for `MetricInput` and `MetricInputMeasure`

* Add `post_aggregation_reference` to `MetricInput` and fix typo in `MetricInputMeasure`

DSI 0.1.0dev7 added `post_aggregation_reference` to the `MetricInput` protocol,
thus we've added it to our implementation in core. Additionally, we had a typo
in a method name in our implementation of `MetricInputMeasure`, ironically
a similar function to the one we've added for `MetricInput`

* Changie doc for upgraded to DSI 0.1.0dev7

* Fix parsing of metric numerator and denominator in schema_yaml_readers

Previously numerator and denominator of a metric were `MetricInputMeasure`s,
now they're `MetricInput`s. Changing the typing isn't enough though.
We have parsing functions in `schema_yaml_readers` which were specifically
parsing the numerator and denominator as if they were `MetricInputMeasure`s.
Thus we had to updating the schema_yaml_readers to parse them as `MetricInput`s.
During this we had some logic in a parsing function `_get_metric_inputs` which
could be abstracted to newly added functions.
2023-06-20 08:08:57 -07:00
Quigley Malcolm
86583a350f Ct-2690 Support dbt-semantic-interfaces 0.1.0dev5 (#7888)
* Upgrade to dbt-semantic-interfaces v0.1.0dev5

This is a fairly simple upgrade. Literally it's just pointing at the
the new versions. The v3 schemas are directly compatible with v5 because
there were no protocol level changes from v3 to v5. All the changers were
updates to tools MetricFlow uses from DSI, not tools that we ourselves
are using in core (yet).

* Add changie doc for DSI version bump
2023-06-16 12:47:55 -07:00
Quigley Malcolm
fafab5d557 [CT-2696] Skip jinia parsing of metric filters (#7885)
* Update metric filters in testing fixtures

I incorrectly wrote the tests such that they didn't include curly
braces, `{{..}}`, around things like `dimension(..)` for filters.
This updates the tests fixtures to have proper filter specifications

* Skip jinja rendering of `filter` key of metrics

Note that `filter` can show up in multiple places: as a root key
on a metric (`metric.filter`), on a metric input (`metric.type_params.metrics[x].filter`),
denominator (`metric.type_params.denominator.filter`), numerator
(`metric.type_params.numerator.filter`), and a metric input measure
(`metric.type_params.measure.filter` and `metric.type_params.measures[x].filter`).
In this commit we skip all of them :)

* Add changie doc for skipping jinja parsing for metric filters

* Update yaml renderer test for metrics
2023-06-15 15:41:31 -07:00
Gerda Shank
39e0c22353 Allow setting packages in dependencies.yml and move dependencies to runtime config (#7857) 2023-06-15 14:41:57 -04:00
colin-rogers-dbt
f767943fb2 Add AdapterRegistered event log message (#7862)
* Add AdapterRegistered event log message

* Add AdapterRegistered to unit test

* make versioning and logging consistent

* make versioning and logging consistent

* add to_version_string

* remove extra equals

* format fire_event
2023-06-14 14:56:47 -07:00
dependabot[bot]
ae97831ebf Bump mypy from 0.981 to 1.0.1 (#7027)
* Bump mypy from 0.981 to 1.0.1

Bumps [mypy](https://github.com/python/mypy) from 0.981 to 1.0.1.
- [Release notes](https://github.com/python/mypy/releases)
- [Commits](https://github.com/python/mypy/compare/v0.981...v1.0.1)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

* upgrade mypy and fix all errors

* fixing some duplicate imports from conflict resolution

* fix mypy errors from merging in main

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com>
2023-06-14 14:31:57 -05:00
Jeremy Cohen
f16bae0ab9 Fix: dbt show --inline with private models (#7838)
* Add functional test

* Check resource_type before DbtReferenceError

* Changelog entry
2023-06-14 12:23:20 -04:00
FishtownBuildBot
b947b2bc7e [Automated] Merged prep-release/1.6.0b4_5260249343 into target main during release process 2023-06-13 16:40:37 -05:00
Github Build Bot
7068688181 Bumping version to 1.6.0b4 and generate changelog 2023-06-13 20:54:08 +00:00
Quigley Malcolm
38c0600982 Update SemanticModel node to match DSI 0.1.0dev3 protocols (#7848)
* Add tests to ensure our semantic layer nodes satisfy the DSI protocols

These tests create runtime checkable versions of the protocols defined in
DSI. Thus we can instantiate instances of our semantic layer nodes and
use `isinstance` to check that they satisfy the protocol. These `runtime_checkable`
versions of the protocols should only exist in testing and should never
be used in the actual package code.

* Update the `Dimension` object of `SemanticModel` node to match DSI protocol

* Make `UnparsedDimension` more strict and update schema readers accordingly

* Update the `Entity` object of `SemanticModel` node to match DSI protocol

* Make `UnparsedEntity` more strict and update schema readers accordingly

* Update the `Measure` object of `SemanticModel` node to match DSI protocol

* Make `UnparsedMeasure` more strict and update schema readers accordingly

* Update the `SemanticModel` node to match DSI protocol

A lot of the additions are helper functions which we don't actually
use in core. This is a known issue. We're in the process of removing
a fair number of them from the DSI protocol spec. However, in the meantime
we need to implement them to satisfy the protocol unfortunately.

* Make `UnparsedSemanticModel` more strict and update schema readers accordingly

* Changie entry for updating SemanticModel node
2023-06-13 13:25:35 -07:00
Jeremy Cohen
83d163add5 Respect column quote config in model contracts (#7537) 2023-06-13 15:32:56 -04:00
Emily Rockman
d46e8855ef Allow ProjectDependency to have extra fields (#7834)
* Allow ProjectDependency to have extra fields

* changelog
2023-06-13 09:47:50 -05:00
mirnawong1
60524c0f8e update adapters url (#7779)
* update adapters url

in response to [docs.getedbt.com pr 3465](https://github.com/dbt-labs/docs.getdbt.com/issues/3465), updating this error message to point to the correct URL, which was recently changed. 

old URL: https://docs.getdbt.com/docs/supported-data-platforms#adapter-installation
new URL: https://docs.getdbt.com/docs/connect-adapters#install-using-the-cli

thank you @dbeatty10 for your 🦅 👀 !

* adding changie entry

* Update .changes/unreleased/Breaking Changes-20230612-161159.yaml

---------

Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com>
2023-06-13 08:51:21 -05:00
Gerda Shank
ca73a2aa15 Use project directory in path selector instead of cwd (#7829)
* Use contextvar to store and get project_root for path selector method

* Changie

* Modify test to check Path selector with project-dir

* Don't set cv_project_root in base task if no config
2023-06-13 09:08:09 -04:00
Emily Rockman
4a833a4272 Move remaining unit tests to /tests directory (#7843)
* moved remaining unit tests to  directory

* fix path

* Delete profiles.yml

* rename remaining test refs
2023-06-13 07:08:28 -05:00
FishtownBuildBot
f9abeca231 Add new index.html and changelog yaml files from dbt-docs (#7836) 2023-06-12 11:55:42 -04:00
Jeremy Cohen
5f9e527768 Rm spaces from NodeType strings (#7842)
* Rm space from NodeType strings

* Add changelog entry
2023-06-12 11:55:07 -04:00
FishtownBuildBot
6f51de4cb5 [Automated] Merged prep-release/1.6.0b3_5215324721 into target main during release process 2023-06-08 15:52:09 -05:00
Github Build Bot
cb64682d33 Bumping version to 1.6.0b3 and generate changelog 2023-06-08 20:15:20 +00:00
Quigley Malcolm
98d1a94b60 Update spec MetricNode for dbt x MetricFlow integration and begin outputting semantic manifest artifact (#7812)
* Refactor MetricNode definition to satisfy DSI Metric protocol

* Fix tests involving metrics to have updated properties

* Update UnparsedMetricNode to match new metric yaml spec

* Update MetricParser for new unparsed and parsed MetricNodes

* Remove `rename_metric_attr`

We're intentionally breaking the spec. There will be a separate tool provided
for migrating from dbt-metrics to dbt x metricflow. This bit of code was renaming
things like `type` to `calculation_method`. This is problematic because `type` is
on the new spec, while `calculation_method` is not. Additionally, since we're
intentionally breaking the spec, this function, `rename_metric_attr`, shouldn't be
used for any property renaming.

* Fix tests for Metrics (1.6) changes

* Regenerated v10 manifest schema and associated functional test artifact state

* Remove no longer needed tests

* Skip / comment out tests for metrics functionality that we'll be implementing later

* Begin outputting semantic manifest artifact on every run

* Drop metrics during upgrade_manifest_json if manifest is v9 or before

* Update properties of `minimal_parsed_metric_dict` to match new metric spec

* Add changie entry for metric node breaking changes

* Add semantic model nodes to semantic manifest
2023-06-08 10:23:36 -07:00
Peter Webb
a89da7ca88 Add SemanticModel Node Type (#7769)
* Add dbt-semantic-interfaces as a dependency

With the integration with MetricFlow we're taking a dependency on
`dbt-semantic-interfaces` which acts as the source of truth for
protocols which MetricFlow and dbt-core need to agree on. Additionally
we're hard pinning to 0.1.0.dev3 for now. We plan on having a less
restrictive specification when dbt-core 1.6 hits GA.

* Add implementations of DSI Metadata protocol to nodes.py

* CT-2521: Initial work on adding new SemanticModel node

* CT-2521: Second rough draft of SemanticModels

* CT-2521: Update schema v10

* CT-2521: Update unit tests for new SemanticModel collection in manifest

* CT-2521: Add changelog entry

* CT-2521: Final touches on initial implementation of SemanticModel parsing

* Change name of Metadata class to reduce potential for confusion

* Remove "Replaceable" inheritance, per review

* CT-2521: Rename internal variables from semantic_models to semantic_nodes

* CT-2521: Update manifest schema to reflect change

---------

Co-authored-by: Quigley Malcolm <quigley.malcolm@dbtlabs.com>
2023-06-08 09:39:04 -04:00
Mike Alfare
2d237828ae ADAP-2: Materialized Views (#7239)
* changie

* ADAP-387: Stub materialized view as a materialization (#7211)

* init attempt at mv and basic forms of helper macros by mixing view and experimental mv sources

* init attempt at mv and basic forms of helper macros by mixing view and experimental mv sources

* remove unneeded return statement, rename directory

* remove unneeded ()

* responding to some pr feedback

* adjusting order of events for mv base work

* move up prexisting drop of backup

* change relatiion type to view to be consistent

* add base test case

* fix jinja exeception message expression, basic test passing

* response to feedback, removeal of refresh infavor of combined create_as, etc.

* swapping to api layer and stratgeies for default implementation (basing off postgres, redshift)

* remove stratgey to limit need for now

* remove unneeded story level changelog entry

* add strategies to condtional in place of old macros

* macro name fix

* rename refresh macro in api level

* align names between postgres and default to same convention

* align names between postgres and default to same convention

* change a create call to full refresh

* pull adapter rename into strategy, add backup_relation as optional arg

* minor typo fix, add intermediate relation to refresh strategy and initial attempt at further conditional logic

* updating to feature main

---------

Co-authored-by: Matthew McKnight <matthew.mcknight@dbtlabs.com>

* ADAP-387: reverting db_api implementation (#7322)

* changie

* init attempt at mv and basic forms of helper macros by mixing view and experimental mv sources

* remove unneeded return statement, rename directory

* remove unneeded ()

* responding to some pr feedback

* adjusting order of events for mv base work

* move up prexisting drop of backup

* change relatiion type to view to be consistent

* add base test case

* fix jinja exeception message expression, basic test passing

* response to feedback, removeal of refresh infavor of combined create_as, etc.

* swapping to api layer and stratgeies for default implementation (basing off postgres, redshift)

* remove stratgey to limit need for now

* remove unneeded story level changelog entry

* add strategies to condtional in place of old macros

* macro name fix

* rename refresh macro in api level

* align names between postgres and default to same convention

* change a create call to full refresh

* pull adapter rename into strategy, add backup_relation as optional arg

* minor typo fix, add intermediate relation to refresh strategy and initial attempt at further conditional logic

* updating to feature main

* removing db_api and strategies directories in favor of matching current materialization setups

* macro name change

* revert to current approach for materializations

* added tests

* added `is_materialized_view` to `BaseRelation`

* updated materialized view stored value to snake case

* typo

* moved materialized view tests into adapter test framework

* add enum to relation for comparison in jinja

---------

Co-authored-by: Mike Alfare <mike.alfare@dbtlabs.com>

* ADAP-391: Add configuration change option (#7272)

* changie

* init attempt at mv and basic forms of helper macros by mixing view and experimental mv sources

* move up pre-existing drop of backup

* change relation type to view to be consistent

* add base test case

* fix jinja exception message expression, basic test passing

* align names between postgres and default to same convention

* init set of Enum for config

* work on initial Enum class for on_configuration_change base it off ConstraintTypes which is also a str based Enum in core

* add on_configuration_change to unit test expected values

* make suggested name change to Enum class

* add on_configuration_change to some integration tests

* add on_configuration_change to expected_manifest to pass functional tests

* added `is_materialized_view` to `BaseRelation`

* updated materialized view stored value to snake case

* moved materialized view tests into adapter test framework

* add alter materialized view macro

* change class name, and config setup

* play with field setup for on_configuration_change

* add method for default selection in enum class

* renamed get_refresh_data_in_materialized_view_sql to align with experimental package

* changed expected values to default string

* added in `on_configuration_change` setting

* change ignore to skip

* updated default option for on_configuration_change on NodeConfig

* removed explicit calls to enum values

* add test setup for testing fail config option

* updated `config_updates` to `configuration_changes` to align with `on_configuration_change` name

* setup configuration change framework

* skipped tests that are expected to fail without adapter implementation

* cleaned up log checks

---------

Co-authored-by: Mike Alfare <mike.alfare@dbtlabs.com>

* ADAP-388: Stub materialized view as a materialization - postgres (#7244)

* move the body of the default macros into the postgres implementation, throw errors if the default is used, indicating that materialized views have not been implemented for that adapter

---------

Co-authored-by: Matthew McKnight <matthew.mcknight@dbtlabs.com>

* ADAP-402: Add configuration change option - postgres (#7334)

* changie

* init attempt at mv and basic forms of helper macros by mixing view and experimental mv sources

* remove unneeded return statement, rename directory

* remove unneeded ()

* responding to some pr feedback

* adjusting order of events for mv base work

* move up prexisting drop of backup

* change relatiion type to view to be consistent

* add base test case

* fix jinja exeception message expression, basic test passing

* added materialized view stubs and test

* response to feedback, removeal of refresh infavor of combined create_as, etc.

* updated postgres to use the new macros structure

* swapping to api layer and stratgeies for default implementation (basing off postgres, redshift)

* remove stratgey to limit need for now

* remove unneeded story level changelog entry

* add strategies to condtional in place of old macros

* macro name fix

* rename refresh macro in api level

* align names between postgres and default to same convention

* change a create call to full refresh

* pull adapter rename into strategy, add backup_relation as optional arg

* minor typo fix, add intermediate relation to refresh strategy and initial attempt at further conditional logic

* init copy of pr 387 to begin 391 implementation

* init set of Enum for config

* work on initial Enum class for on_configuration_change base it off ConstraintTypes which is also a str based Enum in core

* remove postgres-specific materialization in favor of core default materialization

* update db_api to use native types (e.g. str) and avoid direct calls to relation or config, which would alter the run order for all db_api dependencies

* add clarifying comment as to why we have a single test that's expected to fail at the dbt-core layer

* add on_configuration_change to unit test expected values

* make suggested name change to Enum class

* add on_configuration_change to some integretion tests

* add on_configuration_change to expected_manifest to pass functuional tests

* removing db_api and strategies directories in favor of matching current materialization setups

* macro name change

* revert to current approach for materializations

* revert to current approach for materializations

* added tests

* move materialized view logic into the `/materializations` directory in line with `dbt-core`

* moved default macros in `dbt-core` into `dbt-postgres`

* added `is_materialized_view` to `BaseRelation`

* updated materialized view stored value to snake case

* moved materialized view tests into adapter test framework

* updated materialized view tests to use adapter test framework

* add alter materialized view macro

* add alter materialized view macro

* change class name, and config setup

* change class name, and config setup

* play with field setup for on_configuration_change

* add method for default selection in enum class

* renamed get_refresh_data_in_materialized_view_sql to align with experimental package

* changed expected values to default string

* added in `on_configuration_change` setting

* change ignore to skip

* added in `on_configuration_change` setting

* updated default option for on_configuration_change on NodeConfig

* updated default option for on_configuration_change on NodeConfig

* fixed list being passed as string bug

* removed explicit calls to enum values

* removed unneeded test class

* fixed on_configuration_change to be picked up appropriately

* add test setup for testing fail config option

* remove breakpoint, uncomment tests

* update skip scenario to use empty strings

* update skip scenario to avoid using sql at all, remove extra whitespace in some templates

* push up initial addition of indexes for mv macro

* push slight change up

* reverting alt macro and moving the do create_index call to be more in line with other materializations

* Merge branch 'feature/materialized-views/ADAP-2' into feature/materialized-views/ADAP-402

# Conflicts:
#	core/dbt/contracts/graph/model_config.py
#	core/dbt/include/global_project/macros/materializations/models/materialized_view/alter_materialized_view.sql
#	core/dbt/include/global_project/macros/materializations/models/materialized_view/create_materialized_view_as.sql
#	core/dbt/include/global_project/macros/materializations/models/materialized_view/get_materialized_view_configuration_changes.sql
#	core/dbt/include/global_project/macros/materializations/models/materialized_view/materialized_view.sql
#	core/dbt/include/global_project/macros/materializations/models/materialized_view/refresh_materialized_view.sql
#	core/dbt/include/global_project/macros/materializations/models/materialized_view/replace_materialized_view.sql
#	plugins/postgres/dbt/include/postgres/macros/materializations/materialized_view.sql
#	tests/adapter/dbt/tests/adapter/materialized_views/base.py
#	tests/functional/materializations/test_materialized_view.py

* merge feature branch into story branch

* merge feature branch into story branch

* added indexes into the workflow

* fix error in jinja that caused print error

* working on test messaging and skipping tests that might not fit quite into current system

* add drop and show macros for indexes

* add drop and show macros for indexes

* add logic to determine the indexes to create or drop

* pulled index updates through the workflow properly

* convert configuration changes to fixtures, implement index changes into tests

* created Model dataclass for readability, added column to swap index columns for testing

* fixed typo

---------

Co-authored-by: Matthew McKnight <matthew.mcknight@dbtlabs.com>

* ADAP-395: Implement native materialized view DDL (#7336)

* changie

* changie

* init attempt at mv and basic forms of helper macros by mixing view and experimental mv sources

* init attempt at mv and basic forms of helper macros by mixing view and experimental mv sources

* remove unneeded return statement, rename directory

* remove unneeded ()

* responding to some pr feedback

* adjusting order of events for mv base work

* move up prexisting drop of backup

* change relatiion type to view to be consistent

* add base test case

* fix jinja exeception message expression, basic test passing

* added materialized view stubs and test

* response to feedback, removeal of refresh infavor of combined create_as, etc.

* updated postgres to use the new macros structure

* swapping to api layer and stratgeies for default implementation (basing off postgres, redshift)

* remove stratgey to limit need for now

* remove unneeded story level changelog entry

* add strategies to condtional in place of old macros

* macro name fix

* rename refresh macro in api level

* align names between postgres and default to same convention

* align names between postgres and default to same convention

* change a create call to full refresh

* pull adapter rename into strategy, add backup_relation as optional arg

* minor typo fix, add intermediate relation to refresh strategy and initial attempt at further conditional logic

* init copy of pr 387 to begin 391 implementation

* updating to feature main

* updating to feature main

* init set of Enum for config

* work on initial Enum class for on_configuration_change base it off ConstraintTypes which is also a str based Enum in core

* remove postgres-specific materialization in favor of core default materialization

* update db_api to use native types (e.g. str) and avoid direct calls to relation or config, which would alter the run order for all db_api dependencies

* add clarifying comment as to why we have a single test that's expected to fail at the dbt-core layer

* add on_configuration_change to unit test expected values

* make suggested name change to Enum class

* add on_configuration_change to some integretion tests

* add on_configuration_change to expected_manifest to pass functuional tests

* removing db_api and strategies directories in favor of matching current materialization setups

* macro name change

* revert to current approach for materializations

* revert to current approach for materializations

* added tests

* move materialized view logic into the `/materializations` directory in line with `dbt-core`

* moved default macros in `dbt-core` into `dbt-postgres`

* added `is_materialized_view` to `BaseRelation`

* updated materialized view stored value to snake case

* typo

* moved materialized view tests into adapter test framework

* updated materialized view tests to use adapter test framework

* add alter materialized view macro

* add alter materialized view macro

* added basic sql to default macros, added postgres-specific sql for alter scenario, stubbed a test case for index update

* change class name, and config setup

* change class name, and config setup

* play with field setup for on_configuration_change

* add method for default selection in enum class

* renamed get_refresh_data_in_materialized_view_sql to align with experimental package

* changed expected values to default string

* added in `on_configuration_change` setting

* change ignore to skip

* added in `on_configuration_change` setting

* updated default option for on_configuration_change on NodeConfig

* updated default option for on_configuration_change on NodeConfig

* fixed list being passed as string bug

* fixed list being passed as string bug

* removed explicit calls to enum values

* removed explicit calls to enum values

* removed unneeded test class

* fixed on_configuration_change to be picked up appropriately

* add test setup for testing fail config option

* remove breakpoint, uncomment tests

* update skip scenario to use empty strings

* update skip scenario to avoid using sql at all, remove extra whitespace in some templates

* push up initial addition of indexes for mv macro

* push slight change up

* reverting alt macro and moving the do create_index call to be more in line with other materializations

* Merge branch 'feature/materialized-views/ADAP-2' into feature/materialized-views/ADAP-402

# Conflicts:
#	core/dbt/contracts/graph/model_config.py
#	core/dbt/include/global_project/macros/materializations/models/materialized_view/alter_materialized_view.sql
#	core/dbt/include/global_project/macros/materializations/models/materialized_view/create_materialized_view_as.sql
#	core/dbt/include/global_project/macros/materializations/models/materialized_view/get_materialized_view_configuration_changes.sql
#	core/dbt/include/global_project/macros/materializations/models/materialized_view/materialized_view.sql
#	core/dbt/include/global_project/macros/materializations/models/materialized_view/refresh_materialized_view.sql
#	core/dbt/include/global_project/macros/materializations/models/materialized_view/replace_materialized_view.sql
#	plugins/postgres/dbt/include/postgres/macros/materializations/materialized_view.sql
#	tests/adapter/dbt/tests/adapter/materialized_views/base.py
#	tests/functional/materializations/test_materialized_view.py

* merge feature branch into story branch

* merge feature branch into story branch

* added indexes into the workflow

* fix error in jinja that caused print error

* working on test messaging and skipping tests that might not fit quite into current system

* Merge branch 'feature/materialized-views/ADAP-2' into feature/materialized-views/ADAP-395

# Conflicts:
#	core/dbt/include/global_project/macros/materializations/models/materialized_view/get_materialized_view_configuration_changes.sql
#	plugins/postgres/dbt/include/postgres/macros/adapters.sql
#	plugins/postgres/dbt/include/postgres/macros/materializations/materialized_view.sql
#	tests/adapter/dbt/tests/adapter/materialized_views/test_on_configuration_change.py
#	tests/functional/materializations/test_materialized_view.py

* moved postgres implemention into plugin directory

* update index methods to align with the configuration update macro

* added native ddl to postgres macros

* removed extra docstring

* updated references to View, now references MaterializedView

* decomposed materialization into macros

* refactor index create statement parser, add exceptions for unexpected formats

* swapped conditional to check for positive state

* removed skipped test now that materialized view is being used

* return the results and logs of the run so that additional checks can be applied at the adapter level, add check for refresh to a test

* add check for indexes in particular for apply on configuration scenario

* removed extra argument

* add materialized views to get_relations / list_relations

* typos in index change logic

* moved full refresh check inside the build sql step

---------

Co-authored-by: Matthew McKnight <matthew.mcknight@dbtlabs.com>

* removing returns from tests to stop logs from printing

* moved test cases into postgres tests, left non-test functionality in base as new methods or fixtures

* fixed overwrite issue, simplified assertion method

* updated import order to standard

* fixed test import paths

* updated naming convention for proper test collection with the test runner

* still trying to make the test runner happy

* rewrite index updates to use a better source in Postgres

* break out a large test suite as a separate run

* update `skip` and `fail` scenarios with more descriptive results

* typo

* removed call to skip status

* reverting `exceptions_jinja.py`

* added FailFastError back, the right way

* removed PostgresIndex in favor of the already existing PostgresIndexConfig, pulled it into its own file to avoid circular imports

* removed assumed models in method calls, removed odd insert records and replaced with get row count

* fixed index issue, removed some indirection in testing

* made test more readable

* remove the "apply" from the tests and put it on the base as the default

* generalized assertion for reuse with dbt-snowflake, fixed bug in record count utility

* fixed type to be more generic to accommodate adapters with their own relation types

* fixed all the broken index stuff

* updated on_configuration_change to use existing patterns

* updated on_configuration_change to use existing patterns

* reflected update in tests and materialization logic

* reflected update in tests and materialization logic

* reverted the change to create a config object from the option object, using just the option object now

* reverted the change to create a config object from the option object, using just the option object now

* modelled database objects to support monitoring all configuration changes

* updated "skip" to "continue", throw an error on non-implemented macro defaults

* updated "skip" to "continue", throw an error on non-implemented macro defaults

* updated "skip" to "continue", throw an error on non-implemented macro defaults

* updated "skip" to "continue", throw an error on non-implemented macro defaults

* reverted centralized framework, retained a few reusable base classes

* updated names to be more consistent

* readability updates

* added readme specifying that `relation_configs` only supports materialized views for now

---------

Co-authored-by: Matthew McKnight <matthew.mcknight@dbtlabs.com>
Co-authored-by: Matthew McKnight <91097623+McKnight-42@users.noreply.github.com>
2023-06-07 19:19:09 -04:00
Michelle Ark
f4253da72a fix: removing dependency from dependencies.yml (#7743) 2023-06-07 13:48:39 -04:00
Mila Page
919822e583 Adap 496/add test connection mode to debug (#7741)
* --connection-flag

* Standardize the plugin functions used by DebugTask

* Cleanup redundant code and help logic along.

* Add more output tests to add logic coverage and formatting.

* Code review

---------

Co-authored-by: Mila Page <versusfacit@users.noreply.github.com>
2023-06-07 09:56:35 -07:00
Michelle Ark
444c787729 fix error message for empty/None: --warn-error-options handling (#7735) 2023-06-07 12:23:40 -04:00
Michelle Ark
3b63dd9f11 Validate public models are not materialized as ephemeral (#7794) 2023-06-07 12:22:57 -04:00
Kshitij Aranke
84166bf457 Fixes #7551: Create add_from_artifact to populate state_relation field of nodes (#7796) 2023-06-06 15:25:12 -07:00
Michelle Ark
dd445e1fde generalize BaseModelConstraintsRuntimeEnforcement (#7805) 2023-06-06 16:30:50 -04:00
Michelle Ark
6a22ec1b2e Package-namespaced generate_x_name macro resolution (#7509) 2023-06-06 15:05:44 -04:00
Emily Rockman
587bbcbf0d Improve warnings for constraints and materialization types (#7696)
* first pass

* debugging

* regen proto types

* refactor to use warn_supported flag

* PR feedback
2023-06-06 12:50:58 -05:00
Doug Beatty
8e1c4ec116 Fix not equals comparison to be null-safe for adapters/utils tests (#7776)
* Fix names within functional test

* Changelog entry

* Test for implementation of null-safe equals comparison

* Remove duplicated where filter

* Fix null-safe equals comparison

* Fix tests for `concat` and `hash` by using empty strings () instead of `null`

* Remove macro namespace interpolation
2023-06-06 06:11:48 -06:00
Kshitij Aranke
dc35f56baa Fixes #7299: dbt retry (#7763) 2023-06-05 15:51:00 -07:00
Michelle Ark
60d116b5b5 log PublicationArtifactAvailable even when partially parsing & public models unchanged (#7783) 2023-06-05 14:59:38 -04:00
Emily Rockman
4dbc4a41c4 remove entire changes folder (#7766)
Co-authored-by: Kshitij Aranke <kshitij.aranke@dbtlabs.com>
2023-06-05 09:51:30 -05:00
Michelle Ark
89541faec9 force dependency between test models (#7767) 2023-06-05 10:45:59 -04:00
Gerda Shank
79bd98560b Version 0 for model works for latest_version (#7712) 2023-06-05 10:21:39 -04:00
Michelle Ark
7917bd5033 add project_name to manifest metadata (#7754) 2023-06-02 16:16:14 -04:00
Michelle Ark
05b0ebb184 Fix constraint rendering for expressions and foreign key constraint types (#7512) 2023-06-02 15:05:09 -04:00
Quazi Irfan
e1d7a53325 Fix doc link in selector.py (#7755)
* Fix doc link in selector.py

* Ran changie to modify changelog entry
2023-06-02 09:00:58 -05:00
Michelle Ark
7a06d354aa pass optional sql_header to empty subquery sql rendering (#7734) 2023-06-01 14:08:02 -04:00
dave-connors-3
9dd5ab90bf add ability to select models by access (#7739)
* add ability to select models by access

* changie

* Update core/dbt/graph/selector_methods.py
2023-06-01 09:25:59 -04:00
Michelle Ark
45d614533f fix StopIteration error when publication not found (#7710) 2023-05-30 16:50:36 -04:00
Peter Webb
00a531d9d6 Template rendering optimization (#7451)
* CT-2478: Template rendering optimization

* CT-2478: Fix type annotation, and accomodate non-string unit test cases.
2023-05-30 12:48:47 -04:00
Sam Debruyn
fd301a38db Dropped support for Python 3.7 (#7623) 2023-05-30 12:12:57 -04:00
Jeremy Cohen
9c7e01dbca Readd exp_path for config deprecation warnings (#7536) 2023-05-30 12:04:49 -04:00
github-actions[bot]
1ac6df0996 Adding performance modeling for 1.2.0 to refs/heads/main (#7560)
* adding performance baseline for 1.2.0

* Adding newline

---------

Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com>
2023-05-27 22:10:38 -04:00
Gerda Shank
38ca4fce25 Target path should be relative to project dir, rather than current working directory (#7706) 2023-05-26 18:50:38 -04:00
Kshitij Aranke
7e3a6eec96 fix #7300: Enable state for deferral to be separate from state for selectors (#7690) 2023-05-26 13:00:16 -07:00
Emily Rockman
ac16a55c64 Update to reusable workflow for branch testing (#7676)
* fix overlooked node12 case with abandonded marketplace action

* update slack notification

* remove spaces per formatting

* replace with cli dispatch

* move conditional

* add explicit token, temp comment out slack

* add checkout

* checkout teh right branch

* switch to PAT

* add back repo checkout

* manually check workflow status

* fix notification

* swap to reusable workflow

* fix path

* swap permissions

* remove trigger

* fix secrets

* point to main
2023-05-26 14:51:00 -05:00
Doug Beatty
620ca40b85 Add % to adapter suite test cases for persist_docs (#7699)
* Test table/view/column-level comments with `%` symbol

* Test docs block with `%` symbol

* Changelog entry
2023-05-26 12:48:08 -04:00
leahwicz
aa11cf2956 Adding link to 1.5 release notes (#7707) 2023-05-26 08:38:13 -04:00
FishtownBuildBot
feb06e2107 [Automated] Merged prep-release/1.6.0b2_5081502847 into target main during release process 2023-05-25 10:41:03 -05:00
Github Build Bot
a3d40e0abf Bumping version to 1.6.0b2 and generate changelog 2023-05-25 15:06:24 +00:00
Gerda Shank
7c1bd91d0a CT 2590 write pub artifact to log (#7686) 2023-05-24 13:54:58 -04:00
leahwicz
70a132d059 Updating CODEOWNERS to consolidate (#7693)
Consolidating to remove Language and Execution and instead default to the `core-team`
2023-05-24 09:12:25 -04:00
Anis Nasir
1fdebc660b Relaxed the pyyaml dependency to >=5.3. (#7681) 2023-05-24 08:41:38 -04:00
Gerda Shank
0516192d69 CT 2516 ensure that paths in Jinja context flags object are strings (#7678) 2023-05-24 08:24:36 -04:00
github-actions[bot]
f99be58217 Adding performance modeling for 1.4.6 to refs/heads/main (#7532)
* adding performance baseline for 1.4.6

* Add newline

---------

Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com>
2023-05-24 08:17:19 -04:00
github-actions[bot]
3b6222e516 Adding performance modeling for 1.3.0 to refs/heads/main (#7530)
* adding performance baseline for 1.3.0

* Add newline

---------

Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com>
2023-05-24 08:16:30 -04:00
github-actions[bot]
b88e60f8dd Adding performance modeling for 1.4.1 to refs/heads/main (#7527)
* adding performance baseline for 1.4.1

* Add newline

---------

Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com>
2023-05-24 08:16:02 -04:00
github-actions[bot]
9373c4d1e4 Adding performance modeling for 1.3.4 to refs/heads/main (#7525)
* adding performance baseline for 1.3.4

* Add newline

---------

Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com>
2023-05-24 08:15:26 -04:00
dependabot[bot]
0fe3ee8eca Bump ubuntu from 23.04 to 23.10 (#7675)
* Bump ubuntu from 23.04 to 23.10

Bumps ubuntu from 23.04 to 23.10.

---
updated-dependencies:
- dependency-name: ubuntu
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
2023-05-24 08:01:10 -04:00
Sam Debruyn
0d71a32aa2 Include null checks in utils test base (#7672)
* Include null checks in utils test base

* Add tests for the schema test

* Add tests for this macro

---------

Co-authored-by: Mila Page <versusfacit@users.noreply.github.com>
2023-05-23 19:49:48 -07:00
Doug Beatty
0f223663bb Honor --skip-profile-setup parameter when inside an existing project (#7609)
* Honor `--skip-profile-setup` parameter when inside an existing project

* Use project name as the profile name

* Use separate file connections for reading and writing

* Raise a custom exception when no adapters are installed

* Test skipping interactive profile setup when inside a dbt project

* Replace `assert_not_called()` since it does not work

* Verbose CLI argument for skipping profile setup

* Use separate file connections for reading and writing

* Check empty list in a Pythonic manner
2023-05-23 17:56:54 -06:00
Kshitij Aranke
c25d0c9f9c fix #7502: write run_results.json for run operation (#7655) 2023-05-23 14:56:23 -07:00
Peter Webb
4a4b7beeb9 Model Deprecation (#7562)
* CT-2461: Work toward model deprecation

* CT-2461: Remove unneeded conversions

* CT-2461: Fix up unit tests for new fields, correct a couple oversights

* CT-2461: Remaining implementation and tests for model/ref deprecation warnings

* CT-2461: Changelog entry for deprecation warnings

* CT-2461: Refine datetime handling and tests

* CT-2461: Fix up unit test data

* CT-2461: Fix some more unit test data.

* CT-2461: Fix merge issues

* CT-2461: Code review items.

* CT-2461: Improve version -> str conversion
2023-05-23 09:30:32 -04:00
Ian Knox
265e09dc93 Remove DelayedFileHandler (#7661)
* remove DelayedFileHandler

* Changelog

* set_path to no-op

* more no-ops for rpc

* Clearer comments
2023-05-22 16:42:48 -04:00
Mike Alfare
87ea28fe84 break out a large test suite as a separate execution to avoid memory issues on windows CI runs (#7669) 2023-05-19 17:04:35 -04:00
Michelle Ark
af0f786f2e Accept PublicationArtifacts in dbtRunner.invoke (#7656) 2023-05-18 16:42:50 -04:00
David Bloss
50528a009d update used gh actions ahead of node12 deprecation (#7651)
* update used gh actions ahead of node12 deprecation

* replace with valid tag

---------

Co-authored-by: Kshitij Aranke <kshitij.aranke@dbtlabs.com>
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com>
2023-05-17 16:20:26 -05:00
Stu Kilgore
f6e5582370 Add "other" relation to reffable node classes (#7645) 2023-05-17 12:14:16 -05:00
Peter Webb
dea3181d96 Exclude some profile fields from Jinja rendering when they are not valid Jinja. (#7630)
* CT-2583: Exclude some profile fields from Jinja rendering.

* CT-2583: Add functional test.

* CT-2583: Change approach to password jinja detection

* CT-2583: Extract string constant and add additional checks

* CT-2583: Improve unit test coverage
2023-05-17 11:38:48 -04:00
Gerda Shank
5f7ae2fd4c Move node patch method to schema parser patch_node_properties and refactor schema parsing (#7640) 2023-05-16 21:08:04 -04:00
Daniel Reeves
4f249b8652 Add target_path to more cli commands that use it (#7647) 2023-05-16 16:22:59 -05:00
Ian Knox
df23f68dd4 Missed PR fedback (#7642) 2023-05-16 13:50:28 -05:00
Stu Kilgore
4b091cee9e Instantiate Flags class from dict and command name (#7624) 2023-05-16 13:31:23 -05:00
Ian Knox
dcb5acdf29 bugfix: Deps hangs when using relative paths via --project-dir (#7628) 2023-05-16 10:00:23 -05:00
Emily Rockman
7fbeced315 updates for github deprecations (#7614)
* updates for github deprecations

* fix jira file name

* swap out abandonded action

* add quotes to env var

* revert main.yml
2023-05-15 15:28:52 -05:00
Mike Alfare
47e7b1cc80 Feature/drop relation/ct 2581 (#7626)
* changie
* move drop_relation macros into their own file, add scenarios for table, view, and materialized view
2023-05-15 15:51:19 -04:00
FishtownBuildBot
8f998c218e [Automated] Merged prep-release/1.6.0b1_4961250999 into target main during release process 2023-05-12 12:37:41 -05:00
Github Build Bot
41c0797d7a Bumping version to 1.6.0b1 and generate changelog 2023-05-12 17:04:26 +00:00
Michelle Ark
3f2cba0dec add --artifact flag to scripts/collect-artifact-schema (#7599) 2023-05-11 14:31:00 -04:00
Michelle Ark
b60c67d107 add publication artifact to schemas (#7590) 2023-05-11 13:18:35 -04:00
Doug Beatty
630cd3aba0 Allow missing profiles.yml for dbt deps and dbt init (#7546)
* Allow missing `profiles.yml` for `dbt deps` and `dbt init`

* Some commands allow the `--profiles-dir` to not exist

* Remove fix to verify that CI tests work

* Allow missing `profiles.yml` for `dbt deps` and `dbt init`

* CI is not finding any installed adapters

* Remove functional test for `dbt init`
2023-05-10 19:24:04 -06:00
Emily Rockman
05595f5920 Detect breaking changes to constraints in state:modifed (#7476)
* added test that fails

* added new exception

* add partial error checking - needs more specifics

* move contract check under modelnode

* try adding only enforced constraints

* add checks for enforced constraints

* changelog

* add materialization logic

* clean up tests, tweak materializations

* PR feedback

* more PR feedback

* change to tuple
2023-05-10 15:39:26 -05:00
Gerda Shank
29f2cfc48d CT 2510 Throw error for duplicate versioned and non versioned model names (#7577)
* Check for versioned/unversioned duplicates

* Add new exception DuplicateVersionedUnversionedError

* Changie

* Handle packages when finding versioned and unversioned duplicates
2023-05-10 16:16:38 -04:00
Gerda Shank
43d949c5cc CT 2494 check for project level dependency cycles (#7558)
* Raise error if dependent project depends on current project

* Test for project dependency cycle

* Changie
2023-05-10 10:20:03 -04:00
Mike Alfare
58312f1816 CT-2556: pin urllib3 to 1.x (#7574)
* pin urllib3 to 1.x

* changie
2023-05-09 18:36:20 -04:00
Ian Knox
dffbb6a659 Always write run_results.json (#7539) 2023-05-09 13:46:21 -05:00
Kshitij Aranke
272beb21a9 fix #7413: inject sql header in query for show (#7568) 2023-05-09 11:29:12 -07:00
Gerda Shank
d34c511fa5 CT 2552 pin protobuf to >=4.0.0 (#7566)
* Pin protobuf to >=4.0.0

* Changie
2023-05-09 13:10:09 -04:00
github-actions[bot]
2945619eb8 Adding performance modeling for 1.4.0 to refs/heads/main (#7523)
* adding performance baseline for 1.4.0

* Fix formatting

---------

Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com>
2023-05-09 09:14:53 -04:00
Kshitij Aranke
078a83679a fix #7390: push down limit filtering to adapter (#7545) 2023-05-08 22:22:58 -07:00
Kshitij Aranke
881437e890 fix #7273: enable dbt show for seeds (#7544) 2023-05-08 15:08:24 -07:00
Kshitij Aranke
40aca4bc17 fix #7407: print model version in dbt show if specified (#7543) 2023-05-08 14:30:07 -07:00
Michelle Ark
0de046dfbe Allow duplicate refable node names across packages (#7374) 2023-05-08 16:41:37 -04:00
Jeremy Cohen
5a7b73be26 Do not rewrite manifest.json during 'docs serve' command (#7554) 2023-05-08 15:36:07 -04:00
Jeremy Cohen
35f8ceb7f1 Back compat for previous return type of collect_freshness (#7535)
* Back compat for previous retrurn type of 'collect_freshness'

* Test fixups

* PR feedback
2023-05-08 10:11:42 -04:00
Doug Beatty
19d6dab973 Fix inverted --print/--no-print flag (#7524) 2023-05-08 07:20:29 -06:00
leahwicz
810ef7f556 Adding perf testing GHA (#5851)
* Adding perf testing GHA

* Fixing tigger syntax

* Fixing PR creation issue

* Updating testing var

* Remove unneeded branch names

* Fixing branch naming convention

* Standardizing branch name to var

* Consolidating PR jobs

* Updating inputs and making more readable

* Splitting steps up

* Making some updates here to simplify and update

* Remove tab

* Cleaned up testing TODOs before committing

* Fixing spacing

* Fixing spacing issue
2023-05-05 09:41:41 -04:00
Gerda Shank
fd7306643f Initial implementation of cross-project ref (#7276)
* Create publication.py, various Publication classes, Dependency class

* Load dependencies.yml and the corresponding publication file

* Add "public_nodes" and populate ref_lookup

* resolve_ref working

* Add public nodes to parent and child maps

* Bump manifest version and fix tests, use ModelDependsOn

* Split out PublicationArtifact and PublicationConfig, store public_models
separately

* Store dependencies in publication artifact

* change detection of PublicModel for >= python3.10

* Handle removing references for re-processing if publication has changed

* Handle only changed publication artifacts

* Add some logging events

* Remove duplicate nodes from manifest

* refactor relation_from_relation_name

* Remove duplicate writing of manifest.json

* Add public_nodes to flat_graph

* Move some file name constants to core/dbt/constants.py

* Remove "environment" from ProjectDependency. Add
database/schema/identifier to PublicModel. Update TargetNotFound
exception.

* Include external publication dependencies in publication artifact dependencies

* Remove create_from_relation_name, call create_from_node instead

* Change PublicationArtifactChanged message to debug level

* Make write_publication_artifact a function in parser/manifest.py

* Create fixture to create minimal alternate project (just models)

* develop multi project test case
2023-05-03 10:56:40 -04:00
Brice Luu
f1dddaa6e9 Ignore parent tests added edges for build selection (#7431) 2023-04-28 17:01:06 -05:00
Michelle Ark
a7eb89d645 active project > local project in ConfiguredVar (#7441) 2023-04-28 16:17:28 -04:00
Peter Webb
c56a9b2b7f CT-2414: Add graph summaries to target directory output (#7358)
* CT-2414: Add graph summaries to target directory output

* CT-2414: Make graph representation more compact

* CT-2414: Add changelog entry

* CT-2414: Remove temporary diagnostic code.

* CT-2414: Combine graphs into a single file

* CT-2414: Simplify graph summary format.

* CT-2414: Add invocation id to summary, add unit test
2023-04-28 16:00:31 -04:00
Emily Rockman
17a8f462dd Update CODEOWNERS to include the OSS Tooling Guild (#7472)
* Update CODEOWNERS to include the OSS Tooling Guild

* add a few more files
2023-04-28 08:17:53 -05:00
Jeremy Cohen
e3498bdaa5 Remove noisy parse events (#7388)
* Rm noisy parse events

* PR feedback
2023-04-27 18:17:53 +02:00
Gerda Shank
d2f963e20e CT 2483 duplicate depends on nodes (#7455)
* Remove unnecessary "_update_into" methods
2023-04-25 14:10:59 -04:00
Jeremy Cohen
d53bb37186 UX improvements to model versions (#7435)
* Latest version should use un-suffixed alias

* Latest version can be in un-suffixed file

* FYI when unpinned ref to model with prerelease version

* [WIP] Nicer error if versioned ref to unversioned model

* Revert "Latest version should use un-suffixed alias"

This reverts commit 3616c52c1eed7588b9e210e1c957dfda598be550.

* Revert "[WIP] Nicer error if versioned ref to unversioned model"

This reverts commit c9ae4af1cfbd6b7bfc5dcbb445556233eb4bd2c0.

* Define real event for UnpinnedRefNewVersionAvailable

* Update pp test for implicit unsuffixed defined_in

* Add changelog entry

* Fix unit test

* marky feedback

* Add test case for UnpinnedRefNewVersionAvailable event
2023-04-25 19:55:58 +02:00
Michelle Ark
9874f9e004 Fix groupable node partial parsing, raise DbtReferenceError in RuntimeRefResolver (#7438) 2023-04-25 11:09:35 -04:00
Michelle Ark
2739d5f4c4 fix partial parsing of versioned models - schedule child nodes if latest version has been modified (#7439) 2023-04-25 10:30:03 -04:00
Ian Knox
d07603b288 Clear cached statement results when retrieved from ProviderContext and subclasses (#7371) 2023-04-24 12:36:50 -05:00
Jeremy Cohen
723ac9493d Fix .gitignore to take heed of tests/functional/build (#7436) 2023-04-24 19:23:09 +02:00
Stu Kilgore
de75777ede Persist timing info for failed nodes (#7353) 2023-04-24 11:13:41 -05:00
Daniel Reeves
75703c10ee add --target-path to snapshot command (#7419) 2023-04-21 08:29:15 -07:00
Michelle Ark
1722079a43 fix v0 ref resolution and latest_version configuration(#7415) 2023-04-20 12:06:14 -04:00
Michelle Ark
f5aea191d1 add cloud-artifacts as codeowners of /schemas/dbt (#7406) 2023-04-20 12:00:59 -04:00
Chenyu Li
b2418b0634 fix target dir behavior with sources.json (#7412) 2023-04-20 08:02:49 -07:00
Andrey Bystrov
aac034d9ba Empty catalog dbt docs generate (#7202) 2023-04-20 09:57:42 -05:00
Jeremy Cohen
ada8860e48 Update --help text for cache-related parameters (#7389) 2023-04-19 07:31:52 +02:00
Michelle Ark
a87275a4ca [Fix] safe version attribute access in _check_resource_uniqueness (#7376)
safe version attribute access in _check_resource_uniqueness
2023-04-18 14:22:03 -04:00
Michelle Ark
0891aef8d7 [Fix] Typo in statically parsed ref unpacking (#7365)
test statically parsed two-argument ref
2023-04-18 13:06:07 -04:00
Kshitij Aranke
add924221a Add dbt 1.5 to changelog on main (#7403) 2023-04-18 09:17:40 -07:00
github-actions[bot]
ba40d07ea3 [create-pull-request] automated change (#7377)
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: Mike Alfare <mike.alfare@dbtlabs.com>
2023-04-18 11:15:45 -04:00
Michelle Ark
57e9096816 sqlparse <0.4.4 (#7394) 2023-04-18 11:01:38 -04:00
Jeremy Cohen
6fedfe0ece Update docs link in ContractBreakingChangeError message (#7367) 2023-04-18 14:26:45 +02:00
Mike Alfare
121fa5793f dbt.include import issue for ci (#7383)
* separating the installation of the two requirements files
2023-04-18 01:08:40 -04:00
FishtownBuildBot
a88f640395 Cleanup main after cutting new 1.5.latest branch (#7354)
* Bumping version to 1.5.0rc1

* Clean up changelog on main

* fixing linter issues

---------

Co-authored-by: Mike Alfare <mike.alfare@dbtlabs.com>
2023-04-13 23:18:12 -04:00
Michelle Ark
74419b0e86 [Fix] Fix versioned model selection in subdirectories (#7351)
* fix versioned model selection in subdirectories
2023-04-13 10:58:52 -04:00
Jeremy Cohen
2ddf296a8e Adding a new column is not a breaking contract change (#7333)
* Adding a new column is not a breaking contract change

* Add changelog entry

* More structured exception

* same_contract: False if non-breaking changes

* PR feedback: rm build_contract_checksum, more comments
2023-04-13 00:19:46 +02:00
Chenyu Li
6b42a712a8 remove providing profile/project to dbtRunner (#7337) 2023-04-12 12:23:48 -07:00
Benoît Allard
c3230d3374 Allow greater mashumaro versions (#7294)
* Allow greater mashumaro versions

Current 3.5 is working fine.

* Pin mashumaro to 3.6 (released 4 days ago)

* Add changie
2023-04-12 14:40:54 -04:00
Chenyu Li
602535fe71 compile inline query doesn't add node (#7326)
* remove inline node after compile

* add test

* unset inline node
2023-04-12 09:03:55 -07:00
FishtownBuildBot
f9b28bcaed Add new index.html and changelog yaml files from dbt-docs (#7328)
Co-authored-by: Michelle Ark <michelle.ark@dbtlabs.com>
2023-04-12 11:19:49 -04:00
Gerda Shank
922c75344b Add access, has_group, contract_enforced, versioned to run_model (#7309) 2023-04-12 10:56:10 -04:00
Michelle Ark
2caf87c247 Add version selector method (#7330)
Add version selector method: "latest", "prerelease", "old", and "none" selectors supported
2023-04-12 10:48:17 -04:00
Michelle Ark
f2a3535c3f add schema_string_type fixture to test_constraints (#7229) 2023-04-12 10:27:00 -04:00
Peter Webb
a500e60b7f Reset invocation_id Between Commands (#7317)
* CT-2317: Reset invocation id in preflight for each dbt command.

* CT-2317: Add unit test for invocation_id behavior.

* CT-2317: Add changelog entry.

* CT-2317: Modify freshness test to ignore invocation_id

* CT-2317: Assign invocation_id before tracking initialization.

* CT-2317: Fix unit test failures and a bunch of other stuff

* CT-2317: Remove checks which make outdated assumptions about invocation_id being stable between runs

* CT-2317: Review tweak, more unit test fixes.
2023-04-12 09:54:17 -04:00
Michelle Ark
c7ebc8935f model versions (#7287)
model versioning and versioned ref resolution
2023-04-12 09:50:39 -04:00
Neelesh Salian
56f8f8a329 Fix for simple snapshot for seed and test assert (#7296)
* Fix for simple snapshot for seed and test assert

* Bigquery varchar type fix
2023-04-11 19:20:41 -07:00
Doug Beatty
828d723512 Update help text for commands + params in CLI and top-level API (#7176)
* Removed options for `dbt parse`

* Fix misspellings

* Capitalize JSON when appropriate

* Update help text for --write-json/--no-write-json

* Update help text for --config-dir

* Update help text for --resource-types

* Removed decorators for removed dbt parse options

* Remove `--write-manifest` flag from `parse`

* Remove `--parse-only` flag from `compile`

* Update help text for `dbt list --output`

* Standardize on one line per argument

* Factor 3 from 12 Factor CLI Apps

* Update help text for `dbt --version`

* Standardize capitalization of resource types for `dbt build`

* `debug --config-dir` is a boolean flag

* Update help text for `--version-check`

* Specify `-q` as a conventional alias for `--quiet`

* Update help text for `debug --config-dir`

* Update help text for `debug`

* Treat more dense text blobs as binary for `git grep`

* Update help text for `--version-check`

* Update help text for `--defer`

* Update help text for `--indirect-selection`

* Co-locate log colorization with other log settings

* Update help text for `--log-format*`, `--log-level*`, and `--use-colors*`

* Temporarily re-add option for CI tests

* Remove `--parse-only` flag from `show`

* Remove `--write-manifest` flag from `parse` (again)
2023-04-11 17:09:00 -06:00
Robbert Hofman
b450a5754e Snapshot strategies: newline for subquery (#6780)
* Snapshot strategies: newline for subquery

* add changie output

* add test for snapshot ending in comment

* remove import to be flake8 compliant

* add seed import

* add newlines for flake8 compliance

* typo fix

* Fixing up a test, adding a comment or two

* removed un-needed test fixtures

* removed even more un-needed fixtures, collapsed test to single class

* removed errant breakpoint()

* Fix a little typo

---------

Co-authored-by: Ian Knox <ian.knox@dbtlabs.com>
Co-authored-by: Mila Page <67295367+VersusFacit@users.noreply.github.com>
2023-04-11 14:49:36 -07:00
Emily Rockman
2971b9a027 Generalize constraint warnings (#7250)
* CT-1922: Rough in functionality for parsing model level constraints

* CT-1922: (Almost) complete support for model level constraints

* CT-1922: Fix typo affecting correct model constraint parsing.

* CT-1922: Rework base class for model tests for greater simplicity

* CT-1922: Rough in functionality for parsing model level constraints

* CT-1922: Revise unit tests for new model-level constraints property

* CT-1922: (Almost) complete support for model level constraints

* first pass

* implement in core

* add proto

* WIP

* resolve errors in columns_spec_ddl

* changelog

* update comment

* move logic over to python

* rename and use enum

* update default constraint_support dict

* generate new proto definition after conflicts

* reorganize code and break warnings into each constraint

* fix postgres constraint support

* remove breakpoint

* convert constraint support to constant

* update postgres

* add to export

* add to export

* regen proto types file

* standardize names

* put back mypy error

* more naming + add back comma

* add constraint support to model level constraints

* update event message and method signature

* rename method

* CT-1922: Rough in functionality for parsing model level constraints

* CT-1922: Revise unit tests for new model-level constraints property

* CT-1922: (Almost) complete support for model level constraints

* CT-1922: Fix typo affecting correct model constraint parsing.

* CT-1922: Improve whitespace handling

* CT-1922: Render raw constraints to constraint list directly

* make method return consistent

* regenerate proto defn

* update evvent test

* add some code cleanup

---------

Co-authored-by: Peter Allen Webb <peter.webb@dbtlabs.com>
2023-04-11 14:25:55 -05:00
Chenyu Li
3c54959829 remove functions and added time to remove the rest (#7270)
* remove functions and added time to remove the rest

* completely remove lib.py
2023-04-11 11:58:09 -07:00
Stu Kilgore
87e25e8692 Add option to skip relation cache population (#7307) 2023-04-11 13:15:23 -05:00
Peter Webb
6ac5c90a0b CT-1922: Parsing and postgres implementations for model level constraints (#7230)
* CT-1922: Rough in functionality for parsing model level constraints

* CT-1922: Revise unit tests for new model-level constraints property

* CT-1922: (Almost) complete support for model level constraints

* CT-1922: Fix typo affecting correct model constraint parsing.

* CT-1922: Minor code review refinements

* CT-1922: Improve whitespace handling

* CT-1922: Render raw constraints to constraint list directly

* CT-1922: Rework base class for model tests for greater simplicity

* CT-1922: Remove debugging properties. Oops.

* CT-1922: Fix type annotation
2023-04-11 13:59:03 -04:00
Ian Knox
a58fb24e2b Handle duplicated flags better (#7303) 2023-04-11 11:49:15 -05:00
dave-connors-3
9ce593c47f allow resources to be selected by patch path (#7321)
* allow resources to be selected by patch path

* don't ever changie

* use hasattr

* Ignore mypy complaints about patch_path

---------

Co-authored-by: Gerda Shank <gerda@dbtlabs.com>
2023-04-11 12:30:46 -04:00
Jeremy Cohen
c9d4051136 Rebase #7223: improve contracts error message (#7319)
* improved first line of error

* added basic printing of yaml and sql cols as columns

* added changie log

* used listed dictionary as input to match columns

* swapped order of col headers for printing

* used listed dictionary as input to match columns

* removed merge conflict text from file

* Touch-ups

* Update log introspection in functional tests

* Update format_column macro. Case insensitive test

* PR feedback: just data_type, not formatted

---------

Co-authored-by: Kyle Kent <kyle.kent321@gmail.com>
2023-04-11 18:27:23 +02:00
Jeremy Cohen
26f3518cea Update CLI/API docs example, header (#7320)
* Rm CLI/API example

* Update autogen docs header
2023-04-11 18:27:05 +02:00
Ian Knox
49eed67ab0 Parse command should return manifest when invoked via dbtRunner (#7314) 2023-04-11 10:37:56 -05:00
Matteo Ferrando
7a4d3bd2dc fix: avoid dbtRunner default callbacks being shared across instances (#7279)
* fix: avoid dbtRunner default callbacks being shared across instances

* changie
2023-04-11 10:43:17 +02:00
Stu Kilgore
2afb4ccd68 Handle internal exceptions in postflight (#7242) 2023-04-10 17:25:51 -05:00
Gerda Shank
f38d5ad8e2 Add unique_id to CompiledNode and ShowNode logging events (#7308) 2023-04-10 17:09:01 -04:00
Jeremy Cohen
7e1f04c667 Prettify ListRelations message (#7311) 2023-04-10 23:02:01 +02:00
Jeremy Cohen
ef2ba39dcf Add relation info to node_info in logging (#7304)
* Add db, schema, alias, rel_name to node_info

* Add changelog entry

* Default to None

* Nested node_relation message
2023-04-10 22:16:54 +02:00
Tobie Tusing
7045e11aa0 Remove trivial nodes before building subdag (#7194)
* remove trial nodes before building subdag

* add changie

* Update graph.py

remove comment

* further optimize by sorting node search by degree

* change degree to product of in and out degree
2023-04-07 12:09:58 -04:00
Neelesh Salian
a9016c37f5 Fix the base classes of simple snapshot for better import (#7293) 2023-04-07 08:26:38 -07:00
Neelesh Salian
fe62ab8ec5 Moving simple_snapshot test from individual adapters to core adapter zone (#7275)
* Simple snapshot adapter zone

* Class naming

* Class name to Base

* delete extra test file
2023-04-06 18:18:47 -07:00
Gerda Shank
893daedc42 Ensure that same_contract is called for state:modified (#7283) 2023-04-06 15:45:25 -04:00
Doug Beatty
44be13b006 Avoid revoking grants for views when copy_grants=true (#7285)
* Replace hard-coded parameter with intended variable instead

* Changelog entry
2023-04-06 11:37:20 -06:00
Quigley Malcolm
a5131ecc7d CT-2264, CT-2259, CT-1783: Improved event serialization failure handling (#7249)
* Add tests for logging jinja2.Undefined objects

[CT-2259](https://github.com/dbt-labs/dbt-core/issues/7108) identifies
an issue wherein dbt-core 1.0-1.3 raise errors if a jinja2.Undefined
object is attempted to be logged. This generally happened in the form
of `{{ log(undefined_variable, info=True) }}`. This commit adding this
test exists for two reasons
1. Ensure we don't have a regression in this going forward
2. Exist as a commit to be used for backport fixes for dbt-core 1.0-1.3

* Add tests for checking `DBT_ENV_SECRET_`s don't break logging

[CT-1783](https://github.com/dbt-labs/dbt-core/issues/6568) describes
a bug in dbt-core 1.0-1.3 wherein when a `DBT_ENV_SECRET_` all
`{{ log("logging stuff", info=True) }}` invocations break. This commit
adds a test for this for two reasons:
1. Ensure we don't regress to this behavior going forward
2. Act as a base commit for making the backport fixes to dbt-core 1.0-1.3

* Add tests ensuring failed event serialization is handled correctly

[CT-2264](https://github.com/dbt-labs/dbt-core/issues/7113) states
that failed serialization should result in an exception handling path
which will fire another event instead of raising an exception. This is
hard to test perfectly because the exception handling path for
serialization depending on whether pytest is present. If pytest isn't
present, a new event documentation the failed serialization is fired.
If pytest is present, the failed serialization gets raised as an exception.
Thus this added test ensures that the expected exception is raised and
assumes that the correct event will be fired normally.

* Log warning when event serialization fails in `msg_to_dict`

This commit updates the `msg_to_dict` exception handling path to
fire a warning level event instead of raising an exception.
Truthfully, we're not sure if this exception handling path is even
possible to hit. That's because we recently switched from betterproto
to google's protobuf. However, exception path is the subject of
[CT-2264](https://github.com/dbt-labs/dbt-core/issues/7113). Though we
don't think it's actually possible to hit it anymore, we still want
to handle the case if it is.

* Update serialization failure note to be a warn level event in `BaseEvent`

[CT-2264](https://github.com/dbt-labs/dbt-core/issues/7113) wants
logging messages about event serialization failure to be `WARNING`
level events. This does that.

* Add changie info for changes

* Add test to check exception handling of `msg_to_dict`
2023-04-05 19:11:03 -07:00
Kshitij Aranke
ce5d02569f Add kwargs support to dbtRunner (#7274) 2023-04-05 09:40:12 -07:00
Jeremy Cohen
4fc7456000 Small logging fix: stringify datetimes in messages (#7257)
* Stringify datetimes for logging messages

* Add changelog
2023-04-05 15:33:45 +02:00
Kshitij Aranke
28e3412556 New command: dbt show (#7208) 2023-04-04 16:28:22 -07:00
Gerda Shank
86fe510bcf CT 2346 fix read and check versions (#7253) 2023-04-04 18:13:14 -04:00
dependabot[bot]
eaedbd3187 Bump python from 3.11.1-slim-bullseye to 3.11.2-slim-bullseye in /docker (#7196)
* Bump python from 3.11.1-slim-bullseye to 3.11.2-slim-bullseye in /docker

Bumps python from 3.11.1-slim-bullseye to 3.11.2-slim-bullseye.

---
updated-dependencies:
- dependency-name: python
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
2023-04-04 09:31:43 -04:00
Ryan Hill
b31fcc4edf Add double to float types for columns (#6897)
* add double to float types for columns

* add changie item
2023-04-03 14:51:39 -07:00
Emily Rockman
edb5634b9a remove deprecated exception functions (#7251)
* remove deprecated exception functions

* fully remove deprecated exception functions
2023-04-03 09:43:24 -05:00
Gerda Shank
ad21458e10 Fix DepsNotifyUpdatesAvailable logging event (#7262) 2023-04-03 10:11:53 -04:00
dependabot[bot]
622bc43ced Bump black from 22.12.0 to 23.3.0 (#7243)
* Bump black from 22.12.0 to 23.3.0

Bumps [black](https://github.com/psf/black) from 22.12.0 to 23.3.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/22.12.0...23.3.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
2023-03-31 09:55:56 -04:00
FishtownBuildBot
e5d99da0bc [Automated] Merged prep-release/1.5.0b5_4566650829 into target main during release process 2023-03-30 11:53:45 -05:00
Github Build Bot
618499b379 Bumping version to 1.5.0b5 and generate changelog 2023-03-30 16:21:36 +00:00
Gerda Shank
bca361acf9 Remove unnecessary "version" keys in partial parsing pp_dict (#7238) 2023-03-29 16:15:57 -04:00
Ian Knox
567e2ca2be [Missed Commit] Replace tracking events: project_id, adapter_info (#7240) 2023-03-29 14:18:53 -05:00
Ian Knox
474143466f Replace tracking events: project_id, adapter_info (#7231) 2023-03-29 09:41:22 -05:00
Gerda Shank
050161c78f Detect breaking changes to column names and data types in state:modified check (#7216) 2023-03-28 19:54:55 -04:00
Emily Rockman
ab496af1f0 convert contract to dict (#7222)
* first pass at changes before modifying tests

* test updates

* add default

* update manifest

* fix tests

* changelog

* fix unit tests

* rename strict -> enforced

* convert to object

* fix tests

* Update Under the Hood-20230217-105223.yaml

* Update Under the Hood-20230217-105223.yaml

* remove stray breakpoints

* move Contract definition to model_config

* fix contract reference
2023-03-28 17:21:03 -05:00
Doug Beatty
c3c2b27e97 Allow --select and --exclude multiple times (#7169)
* One argument per line

* Tests for multiple `--select` or `--exclude`

* Allow `--select` and `--exclude` multiple times

* Changelog entry

* MultiOption options must be specified with type=tuple or type=ChoiceTuple

* Testing for `--output-keys` and `--resource-type`

* Validate that any new param with `MultiOption` should also have `type=tuple` (or `ChoiceTuple`) and `multiple=True`
2023-03-28 10:50:55 -06:00
Emily Rockman
5789d717ba Enforce model contracts for incremental materializations (#7167)
* first pass

* adding tests

* changelog

* split up tests due to order importance

* update test

* add back comment

* rename base test classes

* move sql

* fix test name

* move sql

* test changes to match main

* organize and cleanup fixtures

* more cleanup of tests
2023-03-28 11:25:24 -05:00
Vyacheslav
14e2c3ec21 CT-2129 Bug: added timeout for dbt version (#7009)
* CT-2129: added timeout for dbt version

* CT-2129: updated CHANGELOG.md

* delete the files added under core/dbt/.changes
2023-03-27 17:13:28 -07:00
Gerda Shank
b718c537a7 Raise upper pin for hologram to 0.0.16 (#7221) 2023-03-27 13:40:25 -04:00
Stu Kilgore
6992151081 Add postflight to handle click exc and exit codes (#7212) 2023-03-27 11:49:37 -05:00
Matthieu Blais
bf5ed39db3 Filter out already existing edges in the subgraph created while building the job queue (#7192) 2023-03-24 13:08:18 -05:00
Gerda Shank
f573870232 Use function for datetime in CommandCompleted and throw error when testing (#7217) 2023-03-24 10:52:01 -04:00
Quigley Malcolm
da4a90aa11 CT-1928: dbtRunner to EventManager callback support (#7214)
* add utility function to EventManager for explicitly adding callbacks

Technically these aren't necessary in their current state. We could instead
have people do `<InstantiatedEventManager>.callbacks.extend(...)` directly.
However, it's not hard to imagine a world wherein extra things need to take
place when a callback is added. Thus abstracting to a utility method
now means that as the implementation of how callbacks are actually added
changes, the invocation to do so can stay the same.

* update `setup_event_logger` to optionally take in callbacks add them to the EventManager

* update preflight decorator to check for and pass along callbacks for event logger setup

* Add `callbacks` to `dbtRunner`

On instantiation of `dbtRunner` one can now provide `callbacks`. These
callbacks are for the `EventLogger`. When `invoke` is called on a `dbtRunner`,
the `callbacks` are added to the cli context object. In the preflight
decorator these callbacks are extracted from the cli context and then
passed to the `setup_event_logger`, finally `setup_event_logger` ensures
the callbacks are added to the global `EVENT_MANAGER`.

* add test to check dbtRunner callbacks get properly set

I believe technically this tests qualifies as more of an integration
test, but no other tests like it currently exist (that I could find
via a cursory search). The `tests/unit/test_dbt_runner.py` seemed like
the most intuitive spot. However, if somewhere else makes sense, I'd be
happy to move it.

* add changie documentation for CT-1928
2023-03-22 17:32:01 -07:00
Mila Page
2cfc386773 Convert simple copy. (#7205)
* Convert simple copy.

* Adjust class names for import.

* adjust test namespacing

* Resolve test error.

---------

Co-authored-by: Mila Page <versusfacit@users.noreply.github.com>
2023-03-22 15:53:07 -07:00
Gerda Shank
ae485f996a CT 1998 use google protobuf to enable more flexible dictionaries (#7190) 2023-03-22 15:59:50 -04:00
Peter Webb
73ff497200 ct-2198: Unify constraints and check_constraints fields (#7130)
* ct-2198: clean up some type names and uses

* CT-2198: Unify constraints and constraints_check properties on columns

* Make mypy version consistently 0.981 (#7134)

* CT 1808 diff based partial parsing (#6873)

* model contracts on models materialized as views (#7120)

* first pass

* rename tests

* fix failing test

* changelog

* fix functional test

* Update core/dbt/parser/base.py

* Update core/dbt/parser/schemas.py

* Create method for env var deprecation (#7086)

* update to allow adapters to change model name resolution in py models (#7115)

* update to allow adapters to change model name resolution in py models

* add changie

* fix newline adds

* move quoting into macro

* use single quotes

* add env DBT_PROJECT_DIR support #6078 (#6659)

Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com>

* Add new index.html and changelog yaml files from dbt-docs (#7141)

* Make version configs optional (#7060)

* [CT-1584] New top level commands: interactive compile (#7008)

Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>

* CT-2198: Add changelog entry

* CT-2198: Fix tests which broke after merge

* CT-2198: Add explicit validation of constraint types w/ unit test

* CT-2198: Move access property, per code review

* CT-2198: Remove a redundant macro

* CT-1298: Rework constraints to be adapter-generated in Python code

* CT-2198: Clarify function name per review

---------

Co-authored-by: Gerda Shank <gerda@dbtlabs.com>
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com>
Co-authored-by: Stu Kilgore <stu.kilgore@dbtlabs.com>
Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com>
Co-authored-by: Leo Schick <67712864+leo-schick@users.noreply.github.com>
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com>
Co-authored-by: FishtownBuildBot <77737458+FishtownBuildBot@users.noreply.github.com>
Co-authored-by: dave-connors-3 <73915542+dave-connors-3@users.noreply.github.com>
Co-authored-by: Kshitij Aranke <kshitij.aranke@dbtlabs.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
2023-03-22 13:08:06 -04:00
Quigley Malcolm
9a7305d43f CT-2049: Add CommandCompleted event (#7180)
* add protobuf message/class for new CommandCompleted event

For [CT-2049](https://github.com/dbt-labs/dbt-core/issues/6878) we
concluded that we wanted a new event type, [CommandCompleted](https://github.com/dbt-labs/dbt-core/issues/6878#issuecomment-1419718606)
with [four (4) values](https://github.com/dbt-labs/dbt-core/issues/6878#issuecomment-1426118283):
which command was run, whether the command succeeded, the timestamp
that the command finished, and how long the command took. This commit
adds the new event proto defition, the auto generated proto_types, and
the instantiatable even type.

* begin emitting CommandCompleted event in the preflight decorator

The [preflight decorator](4186f99b74/core/dbt/cli/requires.py (L19))
runs at the start of every CLI invocation. Thus is a perfect candidate
for emitting the CommandCompleted event. This is noted in the [dicussion
on CT-2049](https://github.com/dbt-labs/dbt-core/issues/6878#issuecomment-1428643539).

* add CommandCompleted event to event unit tests

* Add: changelog entry

* fire CommandCompleted event reguardless of upstream exceptions

Previously, if `--fail-fast` was specified and an issue was run into
or an unhandled issue became an exception, the CommandCompleted event
would not get fired because at this point in the stack we'd be in
exception thrown handling mode. If an exception does reach this point,
we want to still fire the event and also continue to propogate the
exception. Hence the bare `raise` exists to reraise the caught exception

* Update CommandCompleted event to be a `Debug` level event

We don't actually "always" need this event to be logged. Thus we've
updated it to `Debug` level. [Discussion Context](https://github.com/dbt-labs/dbt-core/pull/7180#discussion_r1139281963)
2023-03-22 08:45:11 -07:00
Emily Rockman
ca23148908 Stop ignoring test directory for precommit (#7201)
* reformat test directory to pass formatting checks

* remove test comment
2023-03-22 08:04:13 -05:00
Jeremy Cohen
8225a009b5 Add deprecation warnings for log-path, target-path in dbt_project.yml (#7185)
* Add deprecation warnings for log-path, target-path in dbt_project.yml

* Fix tests/unit/test_events

* Fix failing tests

* PR feedback
2023-03-21 22:31:15 +01:00
Emily Rockman
9605b76178 update workflow to install dev requirements and remove action deprecations (#7203) 2023-03-21 10:41:33 -05:00
Stu Kilgore
137dd9aa1b Deprecate more env vars (#7175) 2023-03-20 11:51:32 -05:00
Gerda Shank
a203fe866a CT 2196, CT2121 constraints column order (#7161) 2023-03-19 19:24:07 -04:00
FishtownBuildBot
4186f99b74 [Automated] Merged prep-release/1.5.0b4_4438341695 into target main during release process 2023-03-16 10:30:09 -05:00
Github Build Bot
6db899eddd Bumping version to 1.5.0b4 and generate changelog 2023-03-16 14:52:41 +00:00
dependabot[bot]
8ea20b4ba2 Update pathspec requirement from <0.11,>=0.9 to >=0.9,<0.12 in /core (#6737)
* Update pathspec requirement from <0.11,>=0.9 to >=0.9,<0.12 in /core

Updates the requirements on [pathspec](https://github.com/cpburnz/python-pathspec) to permit the latest version.
- [Release notes](https://github.com/cpburnz/python-pathspec/releases)
- [Changelog](https://github.com/cpburnz/python-pathspec/blob/master/CHANGES.rst)
- [Commits](https://github.com/cpburnz/python-pathspec/compare/v0.9.0...v0.11.0)

---
updated-dependencies:
- dependency-name: pathspec
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com>
2023-03-16 10:24:04 -04:00
FishtownBuildBot
3f76f82c88 Add new index.html and changelog yaml files from dbt-docs (#7174) 2023-03-15 12:21:01 -05:00
dependabot[bot]
6cbf66db58 Bump python from 3.10.7-slim-bullseye to 3.11.1-slim-bullseye in /docker (#6424)
* Bump python from 3.10.7-slim-bullseye to 3.11.1-slim-bullseye in /docker

Bumps python from 3.10.7-slim-bullseye to 3.11.1-slim-bullseye.

---
updated-dependencies:
- dependency-name: python
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
2023-03-14 23:05:25 -04:00
dependabot[bot]
8cd11b380f Bump black from 22.10.0 to 22.12.0 (#6425)
* Bump black from 22.10.0 to 22.12.0

Bumps [black](https://github.com/psf/black) from 22.10.0 to 22.12.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/22.10.0...22.12.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
2023-03-14 23:04:11 -04:00
Emily Rockman
814eb65d59 Support contract config outside model yaml (#7148)
* first pass

* next pass

* works on local project

* fix tests

* changelog

* remove comment

* update error message

* format message

* update tests
2023-03-14 10:19:14 -05:00
Emily Rockman
f24452a3ab use timezone with no DST in test (#7159) 2023-03-13 12:34:14 -05:00
Kshitij Aranke
30503697f2 [CT-1584] New top level commands: interactive compile (#7008)
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
2023-03-10 17:58:33 -08:00
dave-connors-3
90902689c3 Make version configs optional (#7060) 2023-03-10 10:32:29 -05:00
FishtownBuildBot
5a0e776cff Add new index.html and changelog yaml files from dbt-docs (#7141) 2023-03-09 17:23:38 -08:00
Leo Schick
9368e7a6a1 add env DBT_PROJECT_DIR support #6078 (#6659)
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com>
2023-03-09 10:41:52 -08:00
colin-rogers-dbt
c02ddf8c0e update to allow adapters to change model name resolution in py models (#7115)
* update to allow adapters to change model name resolution in py models

* add changie

* fix newline adds

* move quoting into macro

* use single quotes
2023-03-08 15:33:32 -08:00
Stu Kilgore
64b8a12a42 Create method for env var deprecation (#7086) 2023-03-08 11:07:44 -06:00
Emily Rockman
e895fe9e4b model contracts on models materialized as views (#7120)
* first pass

* rename tests

* fix failing test

* changelog

* fix functional test

* Update core/dbt/parser/base.py

* Update core/dbt/parser/schemas.py
2023-03-07 16:27:25 -06:00
Gerda Shank
8d987521dd CT 1808 diff based partial parsing (#6873) 2023-03-07 16:37:38 -05:00
Gerda Shank
4aafc5ef4a Make mypy version consistently 0.981 (#7134) 2023-03-07 15:12:01 -05:00
Alexander Butler
24ca76ea58 [Feature] Add unix-style fqn wildcard selector method (#6599)
resolves https://github.com/dbt-labs/dbt-core/issues/6598
2023-03-05 06:36:17 -08:00
Michelle Ark
b681908ee2 get_column_schema_from_query macro (#6986)
Add adapter.get_column_schema_from_query
2023-03-03 14:21:22 -05:00
FishtownBuildBot
72076b3fe5 [Automated] Merged prep-release/1.5.0b3_4316612471 into target main during release process 2023-03-02 12:02:33 -06:00
Github Build Bot
0683c59dcd Bumping version to 1.5.0b3 and generate changelog 2023-03-02 17:31:25 +00:00
Stu Kilgore
8019498f09 Remove cli doc generation workflow (#7089) 2023-03-01 10:59:39 -06:00
FishtownBuildBot
6234aec7d2 [Automated] Merged prep-release/1.5.0b2_4298598835 into target main during release process 2023-02-28 18:43:05 -06:00
Github Build Bot
edd8059eb3 Bumping version to 1.5.0b2 and generate changelog 2023-03-01 00:14:39 +00:00
Jeremy Cohen
e3be347768 Roadmap update (Feb 2023) (#7091)
* Init roadmap

* Rework the top paragraph

* Clean-up the whole thing

* Typos and stuff

* Add a missing word

* Fix typo

* Update "when" note

* Next draft

* Propose rename

* Resolve TODOs, still needs a reread

* Being cute

* Another read through

* Fix sentence fragment

---------

Co-authored-by: Florian Eiden <florian.eiden@dbtlabs.com>
2023-02-28 18:48:29 -05:00
FishtownBuildBot
597acf1fa1 Add new index.html and changelog yaml files from dbt-docs (#7092) 2023-02-28 16:11:37 -06:00
Gerda Shank
effa1a0813 Move check of invalid groups earlier in parsing (#7090) 2023-02-28 17:05:20 -05:00
Stu Kilgore
726800be57 Make output_keys param MultiOption (#7068) 2023-02-28 09:29:58 -06:00
Sam Debruyn
8b79747908 fix: add pytz dependency (#7077) 2023-02-28 08:05:28 -06:00
Gerda Shank
ec5d31de0e CT 1993 handle invalid access to private models (#7069) 2023-02-27 17:33:47 -05:00
Emily Rockman
5d61ebbfdb add group for selection logic (#7047)
* add group for selection logic

* changelog

* add more tests

* Update .changes/unreleased/Features-20230224-134533.yaml

* fix broken tests
2023-02-27 15:27:12 -06:00
Gerda Shank
0ef9931d19 CT 1990 add access property to parsed nodes (#7007)
Update manifest schema to version 9
2023-02-24 17:13:09 -05:00
Peter Webb
a2213abbc0 Paw/ct 1844 log params take 2 (#6994)
* CT-1844: Add flags and flag-interaciton tests for more granular log control.

* CT-1844: Apply parameter settings to log configuration

* CT-1844: Fixes per testing

* CT-1844: Type annotation fix.

* Add generated CLI API docs

* CT-1844: Add changelog entry.

* Add generated CLI API docs

* CT-1844: Add new logging parameters to UserConfig

* Add generated CLI API docs

* CT-1844: Code review fixes

* Add generated CLI API docs

---------

Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
2023-02-24 09:18:14 -05:00
Jeremy Cohen
915585c36e Fix regression in semver comparison logic (#7040) 2023-02-23 15:47:22 -08:00
Michelle Ark
5ddd40885e parse group config on groupable nodes (#6965)
parse group config on groupable nodes
2023-02-23 14:17:32 -05:00
Gerda Shank
58d1bccd26 Rename "constraints_enabled" to "contract" (#7002)
* Rename "constraints_enabled" to "contract"

* Changie
2023-02-23 12:16:06 -05:00
Gerda Shank
70c26f5c74 CT 2057 Fix compilation logic for ephemeral nodes (#7023)
* Don't overwrite sql in extra_ctes when compiling (rendering) nodes
2023-02-22 17:24:10 -05:00
dependabot[bot]
ac962a4a31 Bump mypy from 0.971 to 0.981 (#5937)
* Bump mypy from 0.971 to 0.981

Bumps [mypy](https://github.com/python/mypy) from 0.971 to 0.981.
- [Release notes](https://github.com/python/mypy/releases)
- [Commits](https://github.com/python/mypy/compare/v0.971...v0.981)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

* add type to self

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com>
2023-02-22 10:26:55 -06:00
Chenyu Li
bb2d062cc5 remove old pass through method (#7019) 2023-02-21 13:44:11 -08:00
José Duarte
7667784985 Remove trailing slashes from source paths (#6102) (#6179)
* Remove trailing slashes from source paths (#6102)

* Run changie

* Handle mypy complaints

* Revert format
2023-02-21 10:50:00 -06:00
Emily Rockman
05ecfbcc3a rename workflow to match other files (#7000) 2023-02-21 09:40:27 -06:00
Emily Rockman
e06ae97068 fix regex for completeness (#7014) 2023-02-21 08:32:13 -06:00
FishtownBuildBot
ed50877c4f Add new index.html and changelog yaml files from dbt-docs (#7015) 2023-02-20 16:25:14 -06:00
Brice Loustau
6b5e38ee28 Made project version optional (default None) (#6718)
* Made project version optional (default None)

* Changie

* Removed version from fixture dbt_project_yml

* Linting
2023-02-20 08:40:08 -06:00
Jeremy Cohen
63a1bf9adb Update comment for build_manifest_state_check (#7004) 2023-02-19 17:45:59 +01:00
FishtownBuildBot
2c7238fbb4 [Automated] Merged prep-release/1.5.0b1_4206349891 into target main during release process 2023-02-17 12:39:27 -06:00
colin-rogers-dbt
b1d597109f move hook tests to adapter zone (#6952)
* move hook tests to adapter zone

* test update to sql

* fix fixtures

* Add generated CLI API docs

* remove port

* fix column name

* fix missing "host" in profile

* fixture fix

* fixture fix

* fixture/test fix

* Add generated CLI API docs

* fix fixtures

* remove host assumption from test_run_hooks.py

* undo bumpversion.cfg change

* clean up string interpolation and get method

---------

Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
2023-02-17 10:35:21 -08:00
Github Build Bot
7617eece3a Bumping version to 1.5.0b1 and generate changelog 2023-02-17 18:08:59 +00:00
Gerda Shank
8ce92b56d7 CT 2026 test proto serialization of logging events (#6995)
* Fix unit test, functional tests, update structured logging schema check

* Changie

* Remove unneeded cast_to_str
2023-02-17 13:02:51 -05:00
FishtownBuildBot
21fae1c4a4 Add new index.html and changelog yaml files from dbt-docs (#7001) 2023-02-17 09:03:53 -07:00
Chenyu Li
c952d44ec5 fix small issues with flags and logger to support dbt-rpc (#6990) 2023-02-16 16:58:12 -08:00
Doug Beatty
971b38c26b Use IEC standard abbreviations (GiB, TiB, etc) (#6742)
* Use IEC standard abbreviations (GiB, TiB, etc)

* Fix failing test cases

* Remove  accidentally left behind BigQuery-specific methods

* Changelog entry
2023-02-16 14:14:19 -07:00
Michelle Ark
b7884facbf parse group resource (#6921)
parse group resource
2023-02-15 16:49:47 -05:00
Callum McCann
57ce461067 Fixing exposure error (#6934)
* fixing exposure error message

* adding changie
2023-02-15 14:00:20 -06:00
Kshitij Aranke
b1b830643e Revert "Hoist dbt.cli.main into dbt.main namespace (#6939)" (#6987) 2023-02-15 10:11:09 -08:00
Niels Pardon
3cee9d16fa Make use of hashlib.md5() FIPS compliant (#6982)
Signed-off-by: Niels Pardon <par@zurich.ibm.com>
2023-02-15 11:22:25 -05:00
Sung Won Chung
c647706ac2 dbt Constraints / model contracts (#6271)
* start off the blueprints

* test commit

* working snowflake env

* update manifest expectation

* add error handling

* clean up language

* constraints validator

* cleaner example

* better terminal output

* add python error handling

* add to manifest schema

* add to schema config

* clean up comments

* backwards compatible nodeconfig

* remove comments

* clean up more comments

* add changelog

* clarify error message

* constraints list type

* fix grammar

* add global macros

* clearer compile error

* remove comments

* fix tests in this file

* conditional compile errors

* add conditional check in ddl

* add macro to dispatch

* fix regressions in parsed

* fix regressions in manifest tests

* fix manifest test regressions

* fix test_list regressions

* concise data_type terminal error

* remove placeholder function

* fix failed regressions finally

* Revert "Merge branch 'main' of https://github.com/dbt-labs/dbt into dbt-constraints"

This reverts commit 92d2ea7ff5, reversing
changes made to 5d59cc17bc.

* Revert "Revert "Merge branch 'main' of https://github.com/dbt-labs/dbt into dbt-constraints""

This reverts commit 4f747b0bc0.

* remove tmp.csv

* template test plans

* postgres columns spec macro

* schema does not exist error handling

* update postgres adapter

* remove comments

* first passing test

* fix postgres macro

* add more passing tests

* Add generated CLI API docs

* add disabled config test

* column configs match

* test python error handling

* adjust macro with rollback

* start postgres tests

* remove begin commit

* remove begin commit comments

* passing expected compiled sql test

* passing rollback test

* update changelog

* fix artifacts regression

* modularize validator

* PR feedback

* verify database error occurs

* focus on generic outcomes

* fix global macro

* rename to constraints_check

* missed a check rename

* validate at parse time

* raise error for modelparser only

* better spacing in terminal output

* fix test regressions

* fix manifest test regressions

* these are parsing errors now

* fix tests

* test passes in json log format

* add column compile error handling

* update global macros for column handling

* remove TODO

* uppercase columns for consistency

* more specific error handling

* migrate tests

* clean up core tests

* Update core/dbt/include/global_project/macros/materializations/models/table/columns_spec_ddl.sql

For whitespace control in the dbt.log file!

Co-authored-by: dave-connors-3 <73915542+dave-connors-3@users.noreply.github.com>

* Revert "Update core/dbt/include/global_project/macros/materializations/models/table/columns_spec_ddl.sql
"

This reverts commit 59b02980dc.

* update for pre-commit hooks

* update for black formatter

* update for black formatter on all files

* Refactor functional tests

* Fixup formatting

* Dave feedback

* another one - dave

* the hits keep coming

* adjust whitespace

* Light touchup

* Add more flexibility for spark

* Nearly there for spark

* Try regenerating docs

---------

Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: dave-connors-3 <73915542+dave-connors-3@users.noreply.github.com>
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com>
Co-authored-by: Dave Connors <dave.connors@fishtownanalytics.com>
2023-02-15 16:50:39 +01:00
Emily Rockman
7b33ffb1bd point at main (#6973) 2023-02-14 14:17:28 -06:00
Chenyu Li
f38cbc4feb add helper function for load project/profile (#6964) 2023-02-14 08:47:43 -08:00
Gerda Shank
480e0e55c5 Set relation_name in tests at compile time (#6949) 2023-02-14 11:32:27 -05:00
576 changed files with 59687 additions and 22832 deletions

View File

@@ -1,14 +1,18 @@
[bumpversion]
current_version = 1.5.0a1
current_version = 1.6.0b5
parse = (?P<major>[\d]+) # major version number
\.(?P<minor>[\d]+) # minor version number
\.(?P<patch>[\d]+) # patch version number
(((?P<prekind>a|b|rc) # optional pre-release type
?(?P<num>[\d]+?)) # optional pre-release version number
\.?(?P<nightly>[a-z0-9]+)? # optional nightly release indicator
)? # expected matches: `1.5.0`, `1.5.0a1`, `1.5.0a1.dev123457`, expected failures: `1`, `1.5`, `1.5.2-a1`, `text1.5.0`
(?P<prerelease> # optional pre-release - ex: a1, b2, rc25
(?P<prekind>a|b|rc) # pre-release type
(?P<num>[\d]+) # pre-release version number
)?
( # optional nightly release indicator
\.(?P<nightly>dev[0-9]+) # ex: .dev02142023
)? # expected matches: `1.15.0`, `1.5.0a11`, `1.5.0a1.dev123`, `1.5.0.dev123457`, expected failures: `1`, `1.5`, `1.5.2-a1`, `text1.5.0`
serialize =
{major}.{minor}.{patch}{prekind}{num}.{nightly}
{major}.{minor}.{patch}.{nightly}
{major}.{minor}.{patch}{prekind}{num}
{major}.{minor}.{patch}
commit = False

View File

@@ -3,6 +3,7 @@
For information on prior major and minor releases, see their changelogs:
* [1.5](https://github.com/dbt-labs/dbt-core/blob/1.5.latest/CHANGELOG.md)
* [1.4](https://github.com/dbt-labs/dbt-core/blob/1.4.latest/CHANGELOG.md)
* [1.3](https://github.com/dbt-labs/dbt-core/blob/1.3.latest/CHANGELOG.md)
* [1.2](https://github.com/dbt-labs/dbt-core/blob/1.2.latest/CHANGELOG.md)

1
.changes/1.6.0-a1.md Normal file
View File

@@ -0,0 +1 @@
## dbt-core 1.6.0-a1 - April 17, 2023

50
.changes/1.6.0-b1.md Normal file
View File

@@ -0,0 +1,50 @@
## dbt-core 1.6.0-b1 - May 12, 2023
### Features
- Skip catalog generation ([#6980](https://github.com/dbt-labs/dbt-core/issues/6980))
- Publication artifacts and cross-project ref ([#7227](https://github.com/dbt-labs/dbt-core/issues/7227))
- Add graph structure summaries to target path output ([#7357](https://github.com/dbt-labs/dbt-core/issues/7357))
- Allow duplicate manifest node (models, seeds, analyses, snapshots) names across packages ([#7446](https://github.com/dbt-labs/dbt-core/issues/7446))
- Detect breaking changes to enforced constraints ([#7065](https://github.com/dbt-labs/dbt-core/issues/7065))
- Check for project dependency cycles ([#7468](https://github.com/dbt-labs/dbt-core/issues/7468))
### Fixes
- Persist timing info in run results for failed nodes ([#5476](https://github.com/dbt-labs/dbt-core/issues/5476))
- fix typo in unpacking statically parsed ref ([#7364](https://github.com/dbt-labs/dbt-core/issues/7364))
- safe version attribute access in _check_resource_uniqueness ([#7375](https://github.com/dbt-labs/dbt-core/issues/7375))
- Fix dbt command missing target-path param ([# 7411](https://github.com/dbt-labs/dbt-core/issues/ 7411))
- Fix v0 ref resolution ([#7408](https://github.com/dbt-labs/dbt-core/issues/7408))
- Add --target-path to dbt snapshot command. ([#7418](https://github.com/dbt-labs/dbt-core/issues/7418))
- dbt build selection of tests' descendants ([#7289](https://github.com/dbt-labs/dbt-core/issues/7289))
- fix groupable node partial parsing, raise DbtReferenceError at runtime for safety ([#7437](https://github.com/dbt-labs/dbt-core/issues/7437))
- Fix partial parsing of latest_version changes for downstream references ([#7369](https://github.com/dbt-labs/dbt-core/issues/7369))
- Use "add_node" to update depends_on.nodes ([#7453](https://github.com/dbt-labs/dbt-core/issues/7453))
- Fix var precedence in configs: root vars override package vars ([#6705](https://github.com/dbt-labs/dbt-core/issues/6705))
- Fix inverted `--print/--no-print` flag ([#7517](https://github.com/dbt-labs/dbt-core/issues/7517))
- Back-compat for previous return type of 'collect_freshness' macro ([#7489](https://github.com/dbt-labs/dbt-core/issues/7489))
- print model version in dbt show if specified ([#7407](https://github.com/dbt-labs/dbt-core/issues/7407))
- enable dbt show for seeds ([#7273](https://github.com/dbt-labs/dbt-core/issues/7273))
- push down limit filtering to adapter ([#7390](https://github.com/dbt-labs/dbt-core/issues/7390))
- Allow missing `profiles.yml` for `dbt deps` and `dbt init` ([#7511](https://github.com/dbt-labs/dbt-core/issues/7511))
- `run_results.json` is now written after every node completes. ([#7302](https://github.com/dbt-labs/dbt-core/issues/7302))
- Do not rewrite manifest.json during 'docs serve' command ([#7553](https://github.com/dbt-labs/dbt-core/issues/7553))
- Pin protobuf to greater than 4.0.0 ([#7565](https://github.com/dbt-labs/dbt-core/issues/7565))
- inject sql header in query for show ([#7413](https://github.com/dbt-labs/dbt-core/issues/7413))
- Pin urllib3 to ~=1.0 ([#7573](https://github.com/dbt-labs/dbt-core/issues/7573))
- Throw error for duplicated versioned and unversioned models ([#7487](https://github.com/dbt-labs/dbt-core/issues/7487))
### Under the Hood
- Update docs link in ContractBreakingChangeError message ([#7366](https://github.com/dbt-labs/dbt-core/issues/7366))
- Reduce memory footprint of cached statement results. ([#7281](https://github.com/dbt-labs/dbt-core/issues/7281))
- Remove noisy parsing events: GenericTestFileParse, MacroFileParse, Note events for static model parsing ([#6671](https://github.com/dbt-labs/dbt-core/issues/6671))
- Update --help text for cache-related parameters ([#7381](https://github.com/dbt-labs/dbt-core/issues/7381))
- Small UX improvements to model versions: Support defining latest_version in unsuffixed file by default. Notify on unpinned ref when a prerelease version is available. ([#7443](https://github.com/dbt-labs/dbt-core/issues/7443))
### Contributors
- [@AndyBys](https://github.com/AndyBys) ([#6980](https://github.com/dbt-labs/dbt-core/issues/6980))
- [@b-luu](https://github.com/b-luu) ([#7289](https://github.com/dbt-labs/dbt-core/issues/7289))
- [@dwreeves](https://github.com/dwreeves) ([#7418](https://github.com/dbt-labs/dbt-core/issues/7418))
- [@thomasgjerdekog](https://github.com/thomasgjerdekog) ([#7517](https://github.com/dbt-labs/dbt-core/issues/7517))

38
.changes/1.6.0-b2.md Normal file
View File

@@ -0,0 +1,38 @@
## dbt-core 1.6.0-b2 - May 25, 2023
### Features
- Added warnings for model and ref deprecations ([#7433](https://github.com/dbt-labs/dbt-core/issues/7433))
- Update drop_relation macro to allow for configuration of drop statement separately from object name ([#7625](https://github.com/dbt-labs/dbt-core/issues/7625))
- accept publications in dbt.invoke ([#7372](https://github.com/dbt-labs/dbt-core/issues/7372))
### Fixes
- Honor `--skip-profile-setup` parameter when inside an existing project ([#7594](https://github.com/dbt-labs/dbt-core/issues/7594))
- Fix: Relative project paths weren't working with deps ([#7491](https://github.com/dbt-labs/dbt-core/issues/7491))
- Exclude password fields from Jinja rendering. ([#7629](https://github.com/dbt-labs/dbt-core/issues/7629))
- Add --target-path to more CLI subcommands ([#7646](https://github.com/dbt-labs/dbt-core/issues/7646))
- Stringify flag paths for Jinja context ([#7495](https://github.com/dbt-labs/dbt-core/issues/7495))
- write run_results.json for run operation ([#7502](https://github.com/dbt-labs/dbt-core/issues/7502))
### Under the Hood
- Add ability to instantiate Flags class from dict ([#7607](https://github.com/dbt-labs/dbt-core/issues/7607))
- Add other relation to reffable nodes ([#7550](https://github.com/dbt-labs/dbt-core/issues/7550))
- Move node patch method to schema parser patch_node_properties and refactor schema parsing ([#7430](https://github.com/dbt-labs/dbt-core/issues/7430))
- Remove legacy file logger code ([#NA](https://github.com/dbt-labs/dbt-core/issues/NA))
- Break up integration tests as a short term fix for Windows CI runs ([#7668](https://github.com/dbt-labs/dbt-core/issues/7668))
- Include null checks in utils test base ([#7670](https://github.com/dbt-labs/dbt-core/issues/7670))
- Write pub artifact to log ([#7372](https://github.com/dbt-labs/dbt-core/issues/7372))
### Dependencies
- Bump ubuntu from 23.04 to 23.10 ([#7675](https://github.com/dbt-labs/dbt-core/pull/7675))
- ([#7681](https://github.com/dbt-labs/dbt-core/pull/7681))
### Contributors
- [@dradnan89@hotmail.com](https://github.com/dradnan89@hotmail.com) ([#7681](https://github.com/dbt-labs/dbt-core/pull/7681))
- [@dwreeves](https://github.com/dwreeves) ([#7646](https://github.com/dbt-labs/dbt-core/issues/7646))
- [@iknox-fa](https://github.com/iknox-fa) ([#7491](https://github.com/dbt-labs/dbt-core/issues/7491), [#NA](https://github.com/dbt-labs/dbt-core/issues/NA))
- [@sdebruyn](https://github.com/sdebruyn) ([#7670](https://github.com/dbt-labs/dbt-core/issues/7670))
- [@stu-k](https://github.com/stu-k) ([#7607](https://github.com/dbt-labs/dbt-core/issues/7607), [#7550](https://github.com/dbt-labs/dbt-core/issues/7550))

46
.changes/1.6.0-b3.md Normal file
View File

@@ -0,0 +1,46 @@
## dbt-core 1.6.0-b3 - June 08, 2023
### Breaking Changes
- Dropped support for Python 3.7 ([#7082](https://github.com/dbt-labs/dbt-core/issues/7082))
- Switch from dbt-metrics to dbt-semantic-interfaces for MetricNode definitions ([#7500](https://github.com/dbt-labs/dbt-core/issues/7500), [#7404](https://github.com/dbt-labs/dbt-core/issues/7404))
### Features
- Add support for materialized views ([#6911](https://github.com/dbt-labs/dbt-core/issues/6911))
- Optimize template rendering for common parse scenarios ([#7449](https://github.com/dbt-labs/dbt-core/issues/7449))
- nodes in packages respect custom generate_alias_name, generate_schema_name, generate_database_name macro overrides defined in packages ([#7444](https://github.com/dbt-labs/dbt-core/issues/7444))
- Enable state for deferral to be separate from state for selectors ([#7300](https://github.com/dbt-labs/dbt-core/issues/7300))
- add access selection syntax ([#7738](https://github.com/dbt-labs/dbt-core/issues/7738))
- add project_name to manifest metadata ([#7752](https://github.com/dbt-labs/dbt-core/issues/7752))
- dbt retry ([#7299](https://github.com/dbt-labs/dbt-core/issues/7299))
- Revamp debug, add --connection flag. Prepare for future refactors/interface changes. ([#7104](https://github.com/dbt-labs/dbt-core/issues/7104))
- Validate public models are not materialized as ephemeral ([#7226](https://github.com/dbt-labs/dbt-core/issues/7226))
- Added support for parsing and serializaing semantic models ([#7499](https://github.com/dbt-labs/dbt-core/issues/7499), [#7503](https://github.com/dbt-labs/dbt-core/issues/7503))
### Fixes
- Constraint rendering fixes: wrap check expression in parentheses, foreign key 'references', support expression in all constraint types ([#7417](https://github.com/dbt-labs/dbt-core/issues/7417), [#7480](https://github.com/dbt-labs/dbt-core/issues/7480), [#7416](https://github.com/dbt-labs/dbt-core/issues/7416))
- Fix warning messages for deprecated dbt_project.yml configs ([#7424](https://github.com/dbt-labs/dbt-core/issues/7424))
- Add `%` to adapter suite test cases for `persist_docs` ([#7698](https://github.com/dbt-labs/dbt-core/issues/7698))
- Improve warnings for constraints and materialization types ([#7335](https://github.com/dbt-labs/dbt-core/issues/7335))
- Incorrect paths used for "target" and "state" directories ([#7465](https://github.com/dbt-labs/dbt-core/issues/7465))
- fix StopIteration error when publication for project not found ([#7711](https://github.com/dbt-labs/dbt-core/issues/7711))
- Using version 0 works when resolving single model ([#7372](https://github.com/dbt-labs/dbt-core/issues/7372))
- Fix empty --warn-error-options error message ([#7730](https://github.com/dbt-labs/dbt-core/issues/7730))
- send sql header on contract enforcement ([#7714](https://github.com/dbt-labs/dbt-core/issues/7714))
- Fixed doc link in selector.py ([#7533](https://github.com/dbt-labs/dbt-core/issues/7533))
- Fix null-safe equals comparison via `equals` ([#7778](https://github.com/dbt-labs/dbt-core/issues/7778))
- Log PublicationArtifactAvailable even when partially parsing unchanged public models ([#7782](https://github.com/dbt-labs/dbt-core/issues/7782))
- fix RuntimeError when removing project dependency from dependencies.yml ([#7743](https://github.com/dbt-labs/dbt-core/issues/7743))
### Under the Hood
- Fix flaky test for --fail-fast ([#7744](https://github.com/dbt-labs/dbt-core/issues/7744))
- Create `add_from_artifact` to populate `state_relation` field of nodes ([#7551](https://github.com/dbt-labs/dbt-core/issues/7551))
### Contributors
- [@dave-connors-3](https://github.com/dave-connors-3) ([#7738](https://github.com/dbt-labs/dbt-core/issues/7738))
- [@quazi-irfan](https://github.com/quazi-irfan) ([#7533](https://github.com/dbt-labs/dbt-core/issues/7533))
- [@sdebruyn](https://github.com/sdebruyn) ([#7082](https://github.com/dbt-labs/dbt-core/issues/7082))
- [@stu-k](https://github.com/stu-k) ([#7299](https://github.com/dbt-labs/dbt-core/issues/7299), [#7551](https://github.com/dbt-labs/dbt-core/issues/7551))

21
.changes/1.6.0-b4.md Normal file
View File

@@ -0,0 +1,21 @@
## dbt-core 1.6.0-b4 - June 13, 2023
### Fixes
- Respect column 'quote' config in model contracts ([#7370](https://github.com/dbt-labs/dbt-core/issues/7370))
- Fix path selector when using project-dir ([#7819](https://github.com/dbt-labs/dbt-core/issues/7819))
- Allow project dependencies to use miscellaneous keys ([#7497](https://github.com/dbt-labs/dbt-core/issues/7497))
- Updating this error message to point to the correct URL ([#7789](https://github.com/dbt-labs/dbt-core/issues/7789))
- Update SemanticModel node to properly impelment the DSI 0.1.0dev3 SemanticModel protocol spec ([#7833](https://github.com/dbt-labs/dbt-core/issues/7833), [#7827](https://github.com/dbt-labs/dbt-core/issues/7827))
### Docs
- Fix for column tests not rendering on quoted columns ([dbt-docs/#201](https://github.com/dbt-labs/dbt-docs/issues/201))
### Under the Hood
- Rm space from NodeType strings ([#7841](https://github.com/dbt-labs/dbt-core/issues/7841))
### Contributors
- [@drewbanin](https://github.com/drewbanin) ([#201](https://github.com/dbt-labs/dbt-core/issues/201))
- [@mirnawong1](https://github.com/mirnawong1) ([#7789](https://github.com/dbt-labs/dbt-core/issues/7789))

28
.changes/1.6.0-b5.md Normal file
View File

@@ -0,0 +1,28 @@
## dbt-core 1.6.0-b5 - June 22, 2023
### Features
- Enable setting packages in dependencies.yml ([#7372](https://github.com/dbt-labs/dbt-core/issues/7372), [#7736](https://github.com/dbt-labs/dbt-core/issues/7736))
- Add AdapterRegistered event log message ([#7038](https://github.com/dbt-labs/dbt-core/issues/7038))
- Further integrate semantic models into the DAG and partial parsing module ([#7800](https://github.com/dbt-labs/dbt-core/issues/7800))
### Fixes
- Fix regression in `run-operation` to not require the name of the package to run ([#7753](https://github.com/dbt-labs/dbt-core/issues/7753))
- Allow dbt show --inline preview of private models ([#7837](https://github.com/dbt-labs/dbt-core/issues/7837))
- Skip jinja parsing of metric filters ([#7864](https://github.com/dbt-labs/dbt-core/issues/7864))
- Fix a bad implicit string conversion regression in debug --config-dir code. ([#7774](https://github.com/dbt-labs/dbt-core/issues/7774))
- Fix UninstalledPackagesFoundError error message to use correct packages specified path ([#7921](https://github.com/dbt-labs/dbt-core/issues/7921))
- Fix: safe remove of external nodes from nodes.depends_on ([#7924](https://github.com/dbt-labs/dbt-core/issues/7924))
### Under the Hood
- Upgrade to dbt-semantic-interfaces v0.1.0dev5 ([#7853](https://github.com/dbt-labs/dbt-core/issues/7853))
- Refactoring: consolidating public_nodes and nodes ([#7890](https://github.com/dbt-labs/dbt-core/issues/7890))
- Resolve SemanticModel ref is the same way as other refs ([#7822](https://github.com/dbt-labs/dbt-core/issues/7822))
- Move from dbt-semantic-intefaces 0.1.0dev5 to 0.1.0dev7 ([#7898](https://github.com/dbt-labs/dbt-core/issues/7898))
- Don't jinja render packages from dependencies.yml ([#7905](https://github.com/dbt-labs/dbt-core/issues/7905))
### Dependencies
- Bump mypy from 0.981 to 1.0.1 ([#7027](https://github.com/dbt-labs/dbt-core/pull/7027))

View File

@@ -0,0 +1,6 @@
kind: Breaking Changes
body: Dropped support for Python 3.7
time: 2023-05-15T05:31:48.375649+02:00
custom:
Author: sdebruyn
Issue: "7082"

View File

@@ -0,0 +1,6 @@
kind: Breaking Changes
body: Switch from dbt-metrics to dbt-semantic-interfaces for MetricNode definitions
time: 2023-06-07T19:03:09.680189-07:00
custom:
Author: QMalcolm
Issue: 7500 7404

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Updating this error message to point to the correct URL
time: 2023-06-12T16:11:59.181953+01:00
custom:
Author: mirnawong1
Issue: "7789"

View File

@@ -0,0 +1,6 @@
kind: "Dependencies"
body: "Bump mypy from 0.981 to 1.0.1"
time: 2023-02-22T16:28:07.00000Z
custom:
Author: dependabot[bot]
PR: 7027

View File

@@ -0,0 +1,6 @@
kind: "Dependencies"
body: "Bump ubuntu from 23.04 to 23.10"
time: 2023-05-22T00:59:48.00000Z
custom:
Author: dependabot[bot]
PR: 7675

View File

@@ -0,0 +1,5 @@
kind: Dependencies
time: 2023-05-22T21:22:01.336704+02:00
custom:
Author: dradnan89@hotmail.com
PR: "7681"

View File

@@ -0,0 +1,6 @@
kind: Docs
body: Fix for column tests not rendering on quoted columns
time: 2023-05-31T11:54:19.687363-04:00
custom:
Author: drewbanin
Issue: "201"

View File

@@ -0,0 +1,6 @@
kind: Features
body: Skip catalog generation
time: 2023-03-21T21:33:38.513443Z
custom:
Author: AndyBys
Issue: "6980"

View File

@@ -0,0 +1,6 @@
kind: Features
body: Add support for materialized views
time: 2023-03-29T12:03:13.862041-04:00
custom:
Author: mikealfare McKnight-42
Issue: "6911"

View File

@@ -0,0 +1,6 @@
kind: Features
body: Publication artifacts and cross-project ref
time: 2023-04-20T12:47:56.92683-04:00
custom:
Author: gshank
Issue: "7227"

View File

@@ -0,0 +1,6 @@
kind: Features
body: Optimize template rendering for common parse scenarios
time: 2023-04-24T16:36:11.24088-04:00
custom:
Author: peterallenwebb
Issue: "7449"

View File

@@ -0,0 +1,6 @@
kind: Features
body: Add graph structure summaries to target path output
time: 2023-04-25T14:25:22.269051-04:00
custom:
Author: peterallenwebb
Issue: "7357"

View File

@@ -0,0 +1,7 @@
kind: Features
body: Allow duplicate manifest node (models, seeds, analyses, snapshots) names across
packages
time: 2023-04-29T15:50:57.757832-04:00
custom:
Author: MichelleArk
Issue: "7446"

View File

@@ -0,0 +1,6 @@
kind: Features
body: Detect breaking changes to enforced constraints
time: 2023-05-03T10:11:00.617936-05:00
custom:
Author: emmyoop
Issue: "7065"

View File

@@ -0,0 +1,6 @@
kind: Features
body: Check for project dependency cycles
time: 2023-05-09T09:41:47.2-04:00
custom:
Author: gshank
Issue: "7468"

View File

@@ -0,0 +1,7 @@
kind: Features
body: nodes in packages respect custom generate_alias_name, generate_schema_name,
generate_database_name macro overrides defined in packages
time: 2023-05-09T21:29:35.557761-04:00
custom:
Author: michelleark
Issue: "7444"

View File

@@ -0,0 +1,6 @@
kind: Features
body: Added warnings for model and ref deprecations
time: 2023-05-09T23:33:29.679333-04:00
custom:
Author: peterallenwebb
Issue: "7433"

View File

@@ -0,0 +1,7 @@
kind: Features
body: Update drop_relation macro to allow for configuration of drop statement separately
from object name
time: 2023-05-15T12:23:04.177141-04:00
custom:
Author: mikealfare
Issue: "7625"

View File

@@ -0,0 +1,6 @@
kind: Features
body: accept publications in dbt.invoke
time: 2023-05-17T18:56:27.585784-04:00
custom:
Author: MichelleArk
Issue: "7372"

View File

@@ -0,0 +1,6 @@
kind: Features
body: Enable state for deferral to be separate from state for selectors
time: 2023-05-23T22:59:55.920975-07:00
custom:
Author: aranke
Issue: "7300"

View File

@@ -0,0 +1,6 @@
kind: Features
body: add access selection syntax
time: 2023-05-30T16:48:47.740037-05:00
custom:
Author: dave-connors-3
Issue: "7738"

View File

@@ -0,0 +1,6 @@
kind: Features
body: add project_name to manifest metadata
time: 2023-06-01T13:22:23.259448-04:00
custom:
Author: michelleark jtcohen6
Issue: "7752"

View File

@@ -0,0 +1,6 @@
kind: Features
body: dbt retry
time: 2023-06-02T08:33:02.410456-07:00
custom:
Author: stu-k aranke
Issue: "7299"

View File

@@ -0,0 +1,6 @@
kind: Features
body: Revamp debug, add --connection flag. Prepare for future refactors/interface changes.
time: 2023-06-04T02:59:56.28283-07:00
custom:
Author: versusfacit
Issue: 7104

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Fix null-safe equals comparison via `equals`
time: 2023-06-04T08:00:52.537967-06:00
custom:
Author: dbeatty10
Issue: "7778"

View File

@@ -0,0 +1,6 @@
kind: Features
body: ' Validate public models are not materialized as ephemeral'
time: 2023-06-05T22:20:39.382019-04:00
custom:
Author: michelleark
Issue: "7226"

View File

@@ -0,0 +1,6 @@
kind: Features
body: Added support for parsing and serializaing semantic models
time: 2023-06-06T16:53:51.117429-04:00
custom:
Author: peterallenwebb
Issue: 7499 7503

View File

@@ -0,0 +1,6 @@
kind: Features
body: Add AdapterRegistered event log message
time: 2023-06-13T15:15:07.367371-07:00
custom:
Author: colin-rogers-dbt
Issue: "7038"

View File

@@ -0,0 +1,6 @@
kind: Features
body: Enable setting packages in dependencies.yml
time: 2023-06-13T15:19:17.199297-04:00
custom:
Author: gshank
Issue: 7372 7736

View File

@@ -0,0 +1,6 @@
kind: Features
body: Further integrate semantic models into the DAG and partial parsing module
time: 2023-06-16T16:30:45.513314-04:00
custom:
Author: peterallenwebb
Issue: "7800"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Persist timing info in run results for failed nodes
time: 2023-04-13T13:31:57.938847-05:00
custom:
Author: stu-k
Issue: "5476"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: fix typo in unpacking statically parsed ref
time: 2023-04-14T16:36:42.279838-04:00
custom:
Author: MichelleArk
Issue: "7364"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: safe version attribute access in _check_resource_uniqueness
time: 2023-04-18T13:52:57.367108-04:00
custom:
Author: MichelleArk
Issue: "7375"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Fix dbt command missing target-path param
time: 2023-04-19T14:21:50.959786-07:00
custom:
Author: ChenyuLInx
Issue: "\t7411"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Fix v0 ref resolution
time: 2023-04-19T22:09:10.155137-04:00
custom:
Author: MichelleArk
Issue: "7408"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Add --target-path to dbt snapshot command.
time: 2023-04-20T10:42:54.17972-04:00
custom:
Author: dwreeves
Issue: "7418"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: dbt build selection of tests' descendants
time: 2023-04-21T17:24:28.335866975+02:00
custom:
Author: b-luu
Issue: "7289"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: fix groupable node partial parsing, raise DbtReferenceError at runtime for safety
time: 2023-04-24T16:18:43.130637-04:00
custom:
Author: MichelleArk
Issue: "7437"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Fix partial parsing of latest_version changes for downstream references
time: 2023-04-24T16:46:49.721231-04:00
custom:
Author: MichelleArk
Issue: "7369"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Use "add_node" to update depends_on.nodes
time: 2023-04-24T17:34:04.37479-04:00
custom:
Author: gshank
Issue: "7453"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: 'Fix var precedence in configs: root vars override package vars'
time: 2023-04-27T23:07:14.992529-04:00
custom:
Author: MichelleArk
Issue: "6705"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: 'Constraint rendering fixes: wrap check expression in parentheses, foreign key
''references'', support expression in all constraint types'
time: 2023-05-04T14:06:42.545193-04:00
custom:
Author: MichelleArk
Issue: 7417 7480 7416

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Fix inverted `--print/--no-print` flag
time: 2023-05-05T13:25:45.949997-06:00
custom:
Author: dbeatty10 thomasgjerdekog
Issue: "7517"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Back-compat for previous return type of 'collect_freshness' macro
time: 2023-05-06T17:33:15.104953+02:00
custom:
Author: jtcohen6
Issue: "7489"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Fix warning messages for deprecated dbt_project.yml configs
time: 2023-05-06T18:09:00.361961+02:00
custom:
Author: jtcohen6
Issue: "7424"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Respect column 'quote' config in model contracts
time: 2023-05-06T19:18:13.351819+02:00
custom:
Author: jtcohen6
Issue: "7370"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: print model version in dbt show if specified
time: 2023-05-08T04:27:07.9965-07:00
custom:
Author: aranke
Issue: "7407"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: enable dbt show for seeds
time: 2023-05-08T04:49:22.82093-07:00
custom:
Author: aranke
Issue: "7273"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: push down limit filtering to adapter
time: 2023-05-08T06:09:26.455524-07:00
custom:
Author: aranke
Issue: "7390"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Allow missing `profiles.yml` for `dbt deps` and `dbt init`
time: 2023-05-08T07:29:29.873793-06:00
custom:
Author: dbeatty10
Issue: "7511"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: '`run_results.json` is now written after every node completes.'
time: 2023-05-08T09:37:32.809356-05:00
custom:
Author: iknox-fa
Issue: "7302"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Do not rewrite manifest.json during 'docs serve' command
time: 2023-05-08T14:25:18.376379-04:00
custom:
Author: jtcohen6
Issue: "7553"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: inject sql header in query for show
time: 2023-05-09T10:29:32.12947-07:00
custom:
Author: aranke
Issue: "7413"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Pin protobuf to greater than 4.0.0
time: 2023-05-09T12:31:59.019718-04:00
custom:
Author: gshank
Issue: "7565"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Pin urllib3 to ~=1.0
time: 2023-05-09T15:35:30.504674-04:00
custom:
Author: mikealfare
Issue: "7573"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Throw error for duplicated versioned and unversioned models
time: 2023-05-09T16:50:07.530882-04:00
custom:
Author: gshank
Issue: "7487"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Honor `--skip-profile-setup` parameter when inside an existing project
time: 2023-05-11T14:04:41.382181-06:00
custom:
Author: dbeatty10
Issue: "7594"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: 'Fix: Relative project paths weren''t working with deps'
time: 2023-05-15T12:36:54.807413-05:00
custom:
Author: iknox-fa
Issue: "7491"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Exclude password fields from Jinja rendering.
time: 2023-05-15T14:28:51.400321-04:00
custom:
Author: peterallenwebb
Issue: "7629"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Add --target-path to more CLI subcommands
time: 2023-05-16T15:26:44.557072-04:00
custom:
Author: dwreeves
Issue: "7646"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: write run_results.json for run operation
time: 2023-05-22T13:29:24.182612-07:00
custom:
Author: aranke
Issue: "7502"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Stringify flag paths for Jinja context
time: 2023-05-22T13:50:07.897354-04:00
custom:
Author: gshank
Issue: "7495"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Add `%` to adapter suite test cases for `persist_docs`
time: 2023-05-24T16:06:48.477708-06:00
custom:
Author: dbeatty10
Issue: "7698"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Improve warnings for constraints and materialization types
time: 2023-05-25T07:36:51.855641-05:00
custom:
Author: emmyoop
Issue: "7335"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Incorrect paths used for "target" and "state" directories
time: 2023-05-25T16:50:53.718564-04:00
custom:
Author: gshank
Issue: "7465"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: fix StopIteration error when publication for project not found
time: 2023-05-26T15:37:38.952939-04:00
custom:
Author: michelleark
Issue: "7711"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Using version 0 works when resolving single model
time: 2023-05-26T16:47:27.6065-04:00
custom:
Author: gshank
Issue: "7372"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Fix empty --warn-error-options error message
time: 2023-05-30T10:42:28.382804-04:00
custom:
Author: michelleark
Issue: "7730"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: send sql header on contract enforcement
time: 2023-05-31T13:19:19.801391-04:00
custom:
Author: michelleark
Issue: "7714"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Fixed doc link in selector.py
time: 2023-06-01T13:05:49.952-05:00
custom:
Author: quazi-irfan
Issue: "7533"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: Log PublicationArtifactAvailable even when partially parsing unchanged public
models
time: 2023-06-05T12:11:27.739183-04:00
custom:
Author: michelleark
Issue: "7782"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: fix RuntimeError when removing project dependency from dependencies.yml
time: 2023-06-05T12:44:25.978022-04:00
custom:
Author: michelleark
Issue: "7743"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: Fix regression in `run-operation` to not require the name of the package to
run
time: 2023-06-06T14:52:17.38538-07:00
custom:
Author: aranke
Issue: "7753"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Fix path selector when using project-dir
time: 2023-06-08T13:59:52.95775-04:00
custom:
Author: gshank
Issue: "7819"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Allow project dependencies to use miscellaneous keys
time: 2023-06-09T12:19:30.469487-05:00
custom:
Author: emmyoop
Issue: "7497"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Allow dbt show --inline preview of private models
time: 2023-06-09T19:15:46.716379-04:00
custom:
Author: jtcohen6
Issue: "7837"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: Update SemanticModel node to properly impelment the DSI 0.1.0dev3 SemanticModel
protocol spec
time: 2023-06-12T17:58:54.289704-07:00
custom:
Author: QMalcolm
Issue: 7833 7827

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Skip jinja parsing of metric filters
time: 2023-06-15T14:29:49.900201-07:00
custom:
Author: QMalcolm
Issue: "7864"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Fix a bad implicit string conversion regression in debug --config-dir code.
time: 2023-06-21T03:07:33.815966-07:00
custom:
Author: versusfacit
Issue: "7774"

View File

@@ -0,0 +1,7 @@
kind: Fixes
body: Fix UninstalledPackagesFoundError error message to use correct packages specified
path
time: 2023-06-22T10:58:29.576179-04:00
custom:
Author: michelleark
Issue: "7921"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: 'Fix: safe remove of external nodes from nodes.depends_on'
time: 2023-06-22T12:19:07.657855-04:00
custom:
Author: michelleark
Issue: "7924"

View File

@@ -0,0 +1,6 @@
kind: Under the Hood
body: Update docs link in ContractBreakingChangeError message
time: 2023-04-17T11:45:01.005104+02:00
custom:
Author: jtcohen6
Issue: "7366"

View File

@@ -0,0 +1,6 @@
kind: Under the Hood
body: Reduce memory footprint of cached statement results.
time: 2023-04-17T12:27:21.972268-05:00
custom:
Author: iknox-fa
Issue: "7281"

View File

@@ -0,0 +1,7 @@
kind: Under the Hood
body: 'Remove noisy parsing events: GenericTestFileParse, MacroFileParse, Note events
for static model parsing'
time: 2023-04-18T12:12:36.928665+02:00
custom:
Author: jtcohen6
Issue: "6671"

View File

@@ -0,0 +1,6 @@
kind: Under the Hood
body: Update --help text for cache-related parameters
time: 2023-04-18T12:23:23.276693+02:00
custom:
Author: jtcohen6
Issue: "7381"

View File

@@ -0,0 +1,8 @@
kind: Under the Hood
body: 'Small UX improvements to model versions: Support defining latest_version in
unsuffixed file by default. Notify on unpinned ref when a prerelease version is
available. '
time: 2023-04-24T13:53:00.484916+02:00
custom:
Author: jtcohen6
Issue: "7443"

View File

@@ -0,0 +1,6 @@
kind: Under the Hood
body: Add ability to instantiate Flags class from dict
time: 2023-05-15T09:51:16.291352-05:00
custom:
Author: stu-k
Issue: "7607"

View File

@@ -0,0 +1,6 @@
kind: Under the Hood
body: Add other relation to reffable nodes
time: 2023-05-15T15:21:07.808892-05:00
custom:
Author: stu-k
Issue: "7550"

View File

@@ -0,0 +1,7 @@
kind: Under the Hood
body: Move node patch method to schema parser patch_node_properties and refactor schema
parsing
time: 2023-05-16T09:42:41.793503-04:00
custom:
Author: gshank
Issue: "7430"

View File

@@ -0,0 +1,6 @@
kind: Under the Hood
body: Remove legacy file logger code
time: 2023-05-18T11:42:51.825799-05:00
custom:
Author: iknox-fa
Issue: NA

View File

@@ -0,0 +1,6 @@
kind: Under the Hood
body: Break up integration tests as a short term fix for Windows CI runs
time: 2023-05-19T15:30:59.138043-04:00
custom:
Author: mikealfare
Issue: "7668"

View File

@@ -0,0 +1,6 @@
kind: Under the Hood
body: Include null checks in utils test base
time: 2023-05-21T12:57:20.659726+02:00
custom:
Author: sdebruyn
Issue: "7670"

Some files were not shown because too many files have changed in this diff Show More