mirror of
https://github.com/dbt-labs/dbt-core
synced 2025-12-19 00:11:27 +00:00
Compare commits
4 Commits
adding-sem
...
v1.1.0rc2
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
106da05db7 | ||
|
|
09a396a731 | ||
|
|
9c233f27ab | ||
|
|
a09bc28768 |
@@ -1,5 +1,5 @@
|
||||
[bumpversion]
|
||||
current_version = 1.1.0b1
|
||||
current_version = 1.1.0rc2
|
||||
parse = (?P<major>\d+)
|
||||
\.(?P<minor>\d+)
|
||||
\.(?P<patch>\d+)
|
||||
|
||||
48
.changes/1.1.0-rc1.md
Normal file
48
.changes/1.1.0-rc1.md
Normal file
@@ -0,0 +1,48 @@
|
||||
## dbt-core 1.1.0-rc1 - April 12, 2022
|
||||
### Breaking Changes
|
||||
- For adapter plugin maintainers only: Internal adapter methods `set_relations_cache` + `_relations_cache_for_schemas` each take an additional argument, for use with experimental `CACHE_SELECTED_ONLY` config ([#4688](https://github.com/dbt-labs/dbt-core/issues/4688), [#4860](https://github.com/dbt-labs/dbt-core/pull/4860))
|
||||
### Features
|
||||
- Add `--cache_selected_only` flag to cache schema object of selected models only. ([#4688](https://github.com/dbt-labs/dbt-core/issues/4688), [#4860](https://github.com/dbt-labs/dbt-core/pull/4860))
|
||||
- Support custom names for generic tests ([#3348](https://github.com/dbt-labs/dbt-core/issues/3348), [#4898](https://github.com/dbt-labs/dbt-core/pull/4898))
|
||||
- Enable dbt jobs to run downstream models based on fresher sources. Compare the source freshness results between previous and current state. If any source is fresher and/or new in current vs. previous state, dbt will run and test the downstream models in scope. Example command: `dbt build --select source_status:fresher+` ([#4050](https://github.com/dbt-labs/dbt-core/issues/4050), [#4256](https://github.com/dbt-labs/dbt-core/pull/4256))
|
||||
- converting unique key as list tests to new pytest format ([#4882](https://github.com/dbt-labs/dbt-core/issues/4882), [#4958](https://github.com/dbt-labs/dbt-core/pull/4958))
|
||||
- Add a variable called selected_resources in the Jinja context containing a list of all the resources matching the nodes for the --select, --exclude and/or --selector parameters. ([#3471](https://github.com/dbt-labs/dbt-core/issues/3471), [#5001](https://github.com/dbt-labs/dbt-core/pull/5001))
|
||||
- Support the DO_NOT_TRACK environment variable from the consoledonottrack.com initiative ([#3540](https://github.com/dbt-labs/dbt-core/issues/3540), [#5000](https://github.com/dbt-labs/dbt-core/pull/5000))
|
||||
- Add `--no-print` global flag ([#4710](https://github.com/dbt-labs/dbt-core/issues/4710), [#4854](https://github.com/dbt-labs/dbt-core/pull/4854))
|
||||
- add enabled as a source config ([#3662](https://github.com/dbt-labs/dbt-core/issues/3662), [#5008](https://github.com/dbt-labs/dbt-core/pull/5008))
|
||||
### Fixes
|
||||
- Inconsistent timestamps between inserted/updated and deleted rows in snapshots ([#4347](https://github.com/dbt-labs/dbt-core/issues/4347), [#4513](https://github.com/dbt-labs/dbt-core/pull/4513))
|
||||
- Catch more cases to retry package retrieval for deps pointing to the hub. Also start to cache the package requests. ([#4849](https://github.com/dbt-labs/dbt-core/issues/4849), [#4982](https://github.com/dbt-labs/dbt-core/pull/4982))
|
||||
- Make the warning message for a full event deque more descriptive ([#4962](https://github.com/dbt-labs/dbt-core/issues/4962), [#5011](https://github.com/dbt-labs/dbt-core/pull/5011))
|
||||
- Fix hard delete snapshot test ([#4916](https://github.com/dbt-labs/dbt-core/issues/4916), [#5020](https://github.com/dbt-labs/dbt-core/pull/5020))
|
||||
### Docs
|
||||
- Fixed capitalization in UI for exposures of `type: ml` ([#4984](https://github.com/dbt-labs/dbt-core/issues/4984), [#4995](https://github.com/dbt-labs/dbt-core/pull/4995))
|
||||
- List packages and tags in alphabetical order ([#4984](https://github.com/dbt-labs/dbt-core/issues/4984), [#4995](https://github.com/dbt-labs/dbt-core/pull/4995))
|
||||
- Bump jekyll from 3.8.7 to 3.9.0 ([#4984](https://github.com/dbt-labs/dbt-core/issues/4984), [#4995](https://github.com/dbt-labs/dbt-core/pull/4995))
|
||||
- Updated docker README to reflect necessity of using BuildKit ([#4990](https://github.com/dbt-labs/dbt-core/issues/4990), [#5018](https://github.com/dbt-labs/dbt-core/pull/5018))
|
||||
### Under the Hood
|
||||
- add performance regression testing runner without orchestration ([#4021](https://github.com/dbt-labs/dbt-core/issues/4021), [#4602](https://github.com/dbt-labs/dbt-core/pull/4602))
|
||||
- Add Graph Compilation and Adapter Cache tracking ([#4625](https://github.com/dbt-labs/dbt-core/issues/4625), [#4912](https://github.com/dbt-labs/dbt-core/pull/4912))
|
||||
- Create a dbt.tests.adapter release when releasing dbt and postgres ([#4812](https://github.com/dbt-labs/dbt-core/issues/4812), [#4948](https://github.com/dbt-labs/dbt-core/pull/4948))
|
||||
- update docker image to use python 3.10.3 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#4963](https://github.com/dbt-labs/dbt-core/pull/4963))
|
||||
- updates black to 22.3.0 which fixes dependency incompatibility when running with precommit. ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#4972](https://github.com/dbt-labs/dbt-core/pull/4972))
|
||||
- Adds config util for ad-hoc creation of project objs or dicts ([#4808](https://github.com/dbt-labs/dbt-core/issues/4808), [#4981](https://github.com/dbt-labs/dbt-core/pull/4981))
|
||||
- Remove TableComparison and convert existing calls to use dbt.tests.util ([#4778](https://github.com/dbt-labs/dbt-core/issues/4778), [#4986](https://github.com/dbt-labs/dbt-core/pull/4986))
|
||||
- Remove unneeded create_schema in snapshot materialization ([#4742](https://github.com/dbt-labs/dbt-core/issues/4742), [#4993](https://github.com/dbt-labs/dbt-core/pull/4993))
|
||||
- Added .git-blame-ignore-revs file to mask re-formmating commits from git blame ([#5004](https://github.com/dbt-labs/dbt-core/issues/5004), [#5019](https://github.com/dbt-labs/dbt-core/pull/5019))
|
||||
- Convert version tests to pytest ([#5024](https://github.com/dbt-labs/dbt-core/issues/5024), [#5026](https://github.com/dbt-labs/dbt-core/pull/5026))
|
||||
- Updating tests and docs to show that we now support Python 3.10 ([#4974](https://github.com/dbt-labs/dbt-core/issues/4974), [#5025](https://github.com/dbt-labs/dbt-core/pull/5025))
|
||||
- Update --version output and logic ([#4724](https://github.com/dbt-labs/dbt-core/issues/4724), [#5029](https://github.com/dbt-labs/dbt-core/pull/5029))
|
||||
- ([#5033](https://github.com/dbt-labs/dbt-core/issues/5033), [#5032](https://github.com/dbt-labs/dbt-core/pull/5032))
|
||||
|
||||
### Contributors
|
||||
- [@agoblet](https://github.com/agoblet) ([#5000](https://github.com/dbt-labs/dbt-core/pull/5000))
|
||||
- [@anaisvaillant](https://github.com/anaisvaillant) ([#4256](https://github.com/dbt-labs/dbt-core/pull/4256))
|
||||
- [@b-per](https://github.com/b-per) ([#5001](https://github.com/dbt-labs/dbt-core/pull/5001))
|
||||
- [@jonstacks](https://github.com/jonstacks) ([#4995](https://github.com/dbt-labs/dbt-core/pull/4995))
|
||||
- [@kadero](https://github.com/kadero) ([#4513](https://github.com/dbt-labs/dbt-core/pull/4513))
|
||||
- [@karunpoudel](https://github.com/karunpoudel) ([#4860](https://github.com/dbt-labs/dbt-core/pull/4860), [#4860](https://github.com/dbt-labs/dbt-core/pull/4860))
|
||||
- [@matt-winkler](https://github.com/matt-winkler) ([#4256](https://github.com/dbt-labs/dbt-core/pull/4256))
|
||||
- [@pgoslatara](https://github.com/pgoslatara) ([#4995](https://github.com/dbt-labs/dbt-core/pull/4995))
|
||||
- [@poloaraujo](https://github.com/poloaraujo) ([#4854](https://github.com/dbt-labs/dbt-core/pull/4854))
|
||||
- [@sungchun12](https://github.com/sungchun12) ([#4256](https://github.com/dbt-labs/dbt-core/pull/4256))
|
||||
6
.changes/1.1.0-rc2.md
Normal file
6
.changes/1.1.0-rc2.md
Normal file
@@ -0,0 +1,6 @@
|
||||
## dbt-core 1.1.0-rc2 - April 21, 2022
|
||||
### Fixes
|
||||
- Restore ability to utilize `updated_at` for check_cols snapshots ([#5076](https://github.com/dbt-labs/dbt-core/issues/5076), [#5077](https://github.com/dbt-labs/dbt-core/pull/5077))
|
||||
|
||||
### Contributors
|
||||
- [@dbeatty10](https://github.com/dbeatty10) ([#5077](https://github.com/dbt-labs/dbt-core/pull/5077))
|
||||
@@ -5,6 +5,6 @@ body: 'Enable dbt jobs to run downstream models based on fresher sources. Compar
|
||||
models in scope. Example command: `dbt build --select source_status:fresher+` '
|
||||
time: 2022-03-28T13:47:43.750709-05:00
|
||||
custom:
|
||||
Author: sungchun12, matt-winkler, anaisvaillant
|
||||
Author: sungchun12 matt-winkler anaisvaillant
|
||||
Issue: "4050"
|
||||
PR: "4256"
|
||||
7
.changes/1.1.0/Fixes-20220415-112927.yaml
Normal file
7
.changes/1.1.0/Fixes-20220415-112927.yaml
Normal file
@@ -0,0 +1,7 @@
|
||||
kind: Fixes
|
||||
body: Restore ability to utilize `updated_at` for check_cols snapshots
|
||||
time: 2022-04-15T11:29:27.063462-06:00
|
||||
custom:
|
||||
Author: dbeatty10
|
||||
Issue: "5076"
|
||||
PR: "5077"
|
||||
58
CHANGELOG.md
58
CHANGELOG.md
@@ -6,6 +6,64 @@
|
||||
- Do not edit this file directly. This file is auto-generated using [changie](https://github.com/miniscruff/changie). For details on how to document a change, see [the contributing guide](https://github.com/dbt-labs/dbt-core/blob/main/CONTRIBUTING.md#adding-changelog-entry)
|
||||
|
||||
|
||||
## dbt-core 1.1.0-rc2 - April 21, 2022
|
||||
### Fixes
|
||||
- Restore ability to utilize `updated_at` for check_cols snapshots ([#5076](https://github.com/dbt-labs/dbt-core/issues/5076), [#5077](https://github.com/dbt-labs/dbt-core/pull/5077))
|
||||
|
||||
### Contributors
|
||||
- [@dbeatty10](https://github.com/dbeatty10) ([#5077](https://github.com/dbt-labs/dbt-core/pull/5077))
|
||||
|
||||
|
||||
## dbt-core 1.1.0-rc1 - April 12, 2022
|
||||
### Breaking Changes
|
||||
- For adapter plugin maintainers only: Internal adapter methods `set_relations_cache` + `_relations_cache_for_schemas` each take an additional argument, for use with experimental `CACHE_SELECTED_ONLY` config ([#4688](https://github.com/dbt-labs/dbt-core/issues/4688), [#4860](https://github.com/dbt-labs/dbt-core/pull/4860))
|
||||
### Features
|
||||
- Add `--cache_selected_only` flag to cache schema object of selected models only. ([#4688](https://github.com/dbt-labs/dbt-core/issues/4688), [#4860](https://github.com/dbt-labs/dbt-core/pull/4860))
|
||||
- Support custom names for generic tests ([#3348](https://github.com/dbt-labs/dbt-core/issues/3348), [#4898](https://github.com/dbt-labs/dbt-core/pull/4898))
|
||||
- Enable dbt jobs to run downstream models based on fresher sources. Compare the source freshness results between previous and current state. If any source is fresher and/or new in current vs. previous state, dbt will run and test the downstream models in scope. Example command: `dbt build --select source_status:fresher+` ([#4050](https://github.com/dbt-labs/dbt-core/issues/4050), [#4256](https://github.com/dbt-labs/dbt-core/pull/4256))
|
||||
- converting unique key as list tests to new pytest format ([#4882](https://github.com/dbt-labs/dbt-core/issues/4882), [#4958](https://github.com/dbt-labs/dbt-core/pull/4958))
|
||||
- Add a variable called selected_resources in the Jinja context containing a list of all the resources matching the nodes for the --select, --exclude and/or --selector parameters. ([#3471](https://github.com/dbt-labs/dbt-core/issues/3471), [#5001](https://github.com/dbt-labs/dbt-core/pull/5001))
|
||||
- Support the DO_NOT_TRACK environment variable from the consoledonottrack.com initiative ([#3540](https://github.com/dbt-labs/dbt-core/issues/3540), [#5000](https://github.com/dbt-labs/dbt-core/pull/5000))
|
||||
- Add `--no-print` global flag ([#4710](https://github.com/dbt-labs/dbt-core/issues/4710), [#4854](https://github.com/dbt-labs/dbt-core/pull/4854))
|
||||
- add enabled as a source config ([#3662](https://github.com/dbt-labs/dbt-core/issues/3662), [#5008](https://github.com/dbt-labs/dbt-core/pull/5008))
|
||||
### Fixes
|
||||
- Inconsistent timestamps between inserted/updated and deleted rows in snapshots ([#4347](https://github.com/dbt-labs/dbt-core/issues/4347), [#4513](https://github.com/dbt-labs/dbt-core/pull/4513))
|
||||
- Catch more cases to retry package retrieval for deps pointing to the hub. Also start to cache the package requests. ([#4849](https://github.com/dbt-labs/dbt-core/issues/4849), [#4982](https://github.com/dbt-labs/dbt-core/pull/4982))
|
||||
- Make the warning message for a full event deque more descriptive ([#4962](https://github.com/dbt-labs/dbt-core/issues/4962), [#5011](https://github.com/dbt-labs/dbt-core/pull/5011))
|
||||
- Fix hard delete snapshot test ([#4916](https://github.com/dbt-labs/dbt-core/issues/4916), [#5020](https://github.com/dbt-labs/dbt-core/pull/5020))
|
||||
### Docs
|
||||
- Fixed capitalization in UI for exposures of `type: ml` ([#4984](https://github.com/dbt-labs/dbt-core/issues/4984), [#4995](https://github.com/dbt-labs/dbt-core/pull/4995))
|
||||
- List packages and tags in alphabetical order ([#4984](https://github.com/dbt-labs/dbt-core/issues/4984), [#4995](https://github.com/dbt-labs/dbt-core/pull/4995))
|
||||
- Bump jekyll from 3.8.7 to 3.9.0 ([#4984](https://github.com/dbt-labs/dbt-core/issues/4984), [#4995](https://github.com/dbt-labs/dbt-core/pull/4995))
|
||||
- Updated docker README to reflect necessity of using BuildKit ([#4990](https://github.com/dbt-labs/dbt-core/issues/4990), [#5018](https://github.com/dbt-labs/dbt-core/pull/5018))
|
||||
### Under the Hood
|
||||
- add performance regression testing runner without orchestration ([#4021](https://github.com/dbt-labs/dbt-core/issues/4021), [#4602](https://github.com/dbt-labs/dbt-core/pull/4602))
|
||||
- Add Graph Compilation and Adapter Cache tracking ([#4625](https://github.com/dbt-labs/dbt-core/issues/4625), [#4912](https://github.com/dbt-labs/dbt-core/pull/4912))
|
||||
- Create a dbt.tests.adapter release when releasing dbt and postgres ([#4812](https://github.com/dbt-labs/dbt-core/issues/4812), [#4948](https://github.com/dbt-labs/dbt-core/pull/4948))
|
||||
- update docker image to use python 3.10.3 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#4963](https://github.com/dbt-labs/dbt-core/pull/4963))
|
||||
- updates black to 22.3.0 which fixes dependency incompatibility when running with precommit. ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#4972](https://github.com/dbt-labs/dbt-core/pull/4972))
|
||||
- Adds config util for ad-hoc creation of project objs or dicts ([#4808](https://github.com/dbt-labs/dbt-core/issues/4808), [#4981](https://github.com/dbt-labs/dbt-core/pull/4981))
|
||||
- Remove TableComparison and convert existing calls to use dbt.tests.util ([#4778](https://github.com/dbt-labs/dbt-core/issues/4778), [#4986](https://github.com/dbt-labs/dbt-core/pull/4986))
|
||||
- Remove unneeded create_schema in snapshot materialization ([#4742](https://github.com/dbt-labs/dbt-core/issues/4742), [#4993](https://github.com/dbt-labs/dbt-core/pull/4993))
|
||||
- Added .git-blame-ignore-revs file to mask re-formmating commits from git blame ([#5004](https://github.com/dbt-labs/dbt-core/issues/5004), [#5019](https://github.com/dbt-labs/dbt-core/pull/5019))
|
||||
- Convert version tests to pytest ([#5024](https://github.com/dbt-labs/dbt-core/issues/5024), [#5026](https://github.com/dbt-labs/dbt-core/pull/5026))
|
||||
- Updating tests and docs to show that we now support Python 3.10 ([#4974](https://github.com/dbt-labs/dbt-core/issues/4974), [#5025](https://github.com/dbt-labs/dbt-core/pull/5025))
|
||||
- Update --version output and logic ([#4724](https://github.com/dbt-labs/dbt-core/issues/4724), [#5029](https://github.com/dbt-labs/dbt-core/pull/5029))
|
||||
- ([#5033](https://github.com/dbt-labs/dbt-core/issues/5033), [#5032](https://github.com/dbt-labs/dbt-core/pull/5032))
|
||||
|
||||
### Contributors
|
||||
- [@agoblet](https://github.com/agoblet) ([#5000](https://github.com/dbt-labs/dbt-core/pull/5000))
|
||||
- [@anaisvaillant](https://github.com/anaisvaillant) ([#4256](https://github.com/dbt-labs/dbt-core/pull/4256))
|
||||
- [@b-per](https://github.com/b-per) ([#5001](https://github.com/dbt-labs/dbt-core/pull/5001))
|
||||
- [@jonstacks](https://github.com/jonstacks) ([#4995](https://github.com/dbt-labs/dbt-core/pull/4995))
|
||||
- [@kadero](https://github.com/kadero) ([#4513](https://github.com/dbt-labs/dbt-core/pull/4513))
|
||||
- [@karunpoudel](https://github.com/karunpoudel) ([#4860](https://github.com/dbt-labs/dbt-core/pull/4860), [#4860](https://github.com/dbt-labs/dbt-core/pull/4860))
|
||||
- [@matt-winkler](https://github.com/matt-winkler) ([#4256](https://github.com/dbt-labs/dbt-core/pull/4256))
|
||||
- [@pgoslatara](https://github.com/pgoslatara) ([#4995](https://github.com/dbt-labs/dbt-core/pull/4995))
|
||||
- [@poloaraujo](https://github.com/poloaraujo) ([#4854](https://github.com/dbt-labs/dbt-core/pull/4854))
|
||||
- [@sungchun12](https://github.com/sungchun12) ([#4256](https://github.com/dbt-labs/dbt-core/pull/4256))
|
||||
|
||||
|
||||
## dbt-core 1.1.0-b1 - March 22, 2022
|
||||
### Breaking Changes
|
||||
- **Relevant to maintainers of adapter plugins _only_:** The abstractmethods `get_response` and `execute` now only return a `connection.AdapterReponse` in type hints. (Previously, they could return a string.) We encourage you to update your methods to return an object of class `AdapterResponse`, or implement a subclass specific to your adapter ([#4499](https://github.com/dbt-labs/dbt-core/issues/4499), [#4869](https://github.com/dbt-labs/dbt-core/pull/4869))
|
||||
|
||||
@@ -132,7 +132,7 @@
|
||||
{% set check_cols_config = config['check_cols'] %}
|
||||
{% set primary_key = config['unique_key'] %}
|
||||
{% set invalidate_hard_deletes = config.get('invalidate_hard_deletes', false) %}
|
||||
{% set updated_at = snapshot_get_time() %}
|
||||
{% set updated_at = config.get('updated_at', snapshot_get_time()) %}
|
||||
|
||||
{% set column_added = false %}
|
||||
|
||||
|
||||
@@ -273,13 +273,15 @@ def check_relation_types(adapter, relation_to_type):
|
||||
# by doing a separate call for each set of tables/relations.
|
||||
# Wraps check_relations_equal_with_relations by creating relations
|
||||
# from the list of names passed in.
|
||||
def check_relations_equal(adapter, relation_names):
|
||||
def check_relations_equal(adapter, relation_names, compare_snapshot_cols=False):
|
||||
if len(relation_names) < 2:
|
||||
raise TestProcessingException(
|
||||
"Not enough relations to compare",
|
||||
)
|
||||
relations = [relation_from_name(adapter, name) for name in relation_names]
|
||||
return check_relations_equal_with_relations(adapter, relations)
|
||||
return check_relations_equal_with_relations(
|
||||
adapter, relations, compare_snapshot_cols=compare_snapshot_cols
|
||||
)
|
||||
|
||||
|
||||
# This can be used when checking relations in different schemas, by supplying
|
||||
@@ -288,16 +290,17 @@ def check_relations_equal(adapter, relation_names):
|
||||
# adapter.get_columns_in_relation
|
||||
# adapter.get_rows_different_sql
|
||||
# adapter.execute
|
||||
def check_relations_equal_with_relations(adapter, relations):
|
||||
def check_relations_equal_with_relations(adapter, relations, compare_snapshot_cols=False):
|
||||
|
||||
with get_connection(adapter):
|
||||
basis, compares = relations[0], relations[1:]
|
||||
# Skip columns starting with "dbt_" because we don't want to
|
||||
# compare those, since they are time sensitive
|
||||
# (unless comparing "dbt_" snapshot columns is explicitly enabled)
|
||||
column_names = [
|
||||
c.name
|
||||
for c in adapter.get_columns_in_relation(basis)
|
||||
if not c.name.lower().startswith("dbt_")
|
||||
if not c.name.lower().startswith("dbt_") or compare_snapshot_cols
|
||||
]
|
||||
|
||||
for relation in compares:
|
||||
|
||||
@@ -234,5 +234,5 @@ def _get_adapter_plugin_names() -> Iterator[str]:
|
||||
yield plugin_name
|
||||
|
||||
|
||||
__version__ = "1.1.0b1"
|
||||
__version__ = "1.1.0rc2"
|
||||
installed = get_installed_version()
|
||||
|
||||
@@ -273,12 +273,12 @@ def parse_args(argv=None):
|
||||
parser.add_argument("adapter")
|
||||
parser.add_argument("--title-case", "-t", default=None)
|
||||
parser.add_argument("--dependency", action="append")
|
||||
parser.add_argument("--dbt-core-version", default="1.1.0b1")
|
||||
parser.add_argument("--dbt-core-version", default="1.1.0rc2")
|
||||
parser.add_argument("--email")
|
||||
parser.add_argument("--author")
|
||||
parser.add_argument("--url")
|
||||
parser.add_argument("--sql", action="store_true")
|
||||
parser.add_argument("--package-version", default="1.1.0b1")
|
||||
parser.add_argument("--package-version", default="1.1.0rc2")
|
||||
parser.add_argument("--project-version", default="1.0")
|
||||
parser.add_argument("--no-dependency", action="store_false", dest="set_dependency")
|
||||
parsed = parser.parse_args()
|
||||
|
||||
@@ -25,7 +25,7 @@ with open(os.path.join(this_directory, "README.md")) as f:
|
||||
|
||||
|
||||
package_name = "dbt-core"
|
||||
package_version = "1.1.0b1"
|
||||
package_version = "1.1.0rc2"
|
||||
description = """With dbt, data analysts and engineers can build analytics \
|
||||
the way engineers build applications."""
|
||||
|
||||
|
||||
@@ -14,12 +14,12 @@ FROM --platform=$build_for python:3.10.3-slim-bullseye as base
|
||||
# N.B. The refs updated automagically every release via bumpversion
|
||||
# N.B. dbt-postgres is currently found in the core codebase so a value of dbt-core@<some_version> is correct
|
||||
|
||||
ARG dbt_core_ref=dbt-core@v1.1.0b1
|
||||
ARG dbt_postgres_ref=dbt-core@v1.1.0b1
|
||||
ARG dbt_redshift_ref=dbt-redshift@v1.0.0
|
||||
ARG dbt_bigquery_ref=dbt-bigquery@v1.0.0
|
||||
ARG dbt_snowflake_ref=dbt-snowflake@v1.0.0
|
||||
ARG dbt_spark_ref=dbt-spark@v1.0.0
|
||||
ARG dbt_core_ref=dbt-core@v1.1.0rc2
|
||||
ARG dbt_postgres_ref=dbt-core@v1.1.0rc2
|
||||
ARG dbt_redshift_ref=dbt-redshift@v1.1.0rc2
|
||||
ARG dbt_bigquery_ref=dbt-bigquery@v1.1.0rc2
|
||||
ARG dbt_snowflake_ref=dbt-snowflake@v1.1.0rc2
|
||||
ARG dbt_spark_ref=dbt-spark@v1.1.0rc2
|
||||
# special case args
|
||||
ARG dbt_spark_version=all
|
||||
ARG dbt_third_party
|
||||
|
||||
@@ -1 +1 @@
|
||||
version = "1.1.0b1"
|
||||
version = "1.1.0rc2"
|
||||
|
||||
@@ -41,7 +41,7 @@ def _dbt_psycopg2_name():
|
||||
|
||||
|
||||
package_name = "dbt-postgres"
|
||||
package_version = "1.1.0b1"
|
||||
package_version = "1.1.0rc2"
|
||||
description = """The postgres adpter plugin for dbt (data build tool)"""
|
||||
|
||||
this_directory = os.path.abspath(os.path.dirname(__file__))
|
||||
|
||||
@@ -1 +1 @@
|
||||
version = "1.1.0b1"
|
||||
version = "1.1.0rc2"
|
||||
|
||||
@@ -20,7 +20,7 @@ except ImportError:
|
||||
|
||||
|
||||
package_name = "dbt-tests-adapter"
|
||||
package_version = "1.1.0b1"
|
||||
package_version = "1.1.0rc2"
|
||||
description = """The dbt adapter tests for adapter plugins"""
|
||||
|
||||
this_directory = os.path.abspath(os.path.dirname(__file__))
|
||||
|
||||
@@ -0,0 +1,113 @@
|
||||
import pytest
|
||||
from dbt.tests.util import run_dbt, check_relations_equal
|
||||
|
||||
snapshot_sql = """
|
||||
{% snapshot snapshot_check_cols_updated_at_actual %}
|
||||
{{
|
||||
config(
|
||||
target_database=database,
|
||||
target_schema=schema,
|
||||
unique_key='id',
|
||||
strategy='check',
|
||||
check_cols='all',
|
||||
updated_at="'" ~ var("updated_at") ~ "'::timestamp",
|
||||
)
|
||||
}}
|
||||
|
||||
{% if var('version') == 1 %}
|
||||
|
||||
select 'a' as id, 10 as counter, '2016-01-01T00:00:00Z'::timestamp as timestamp_col union all
|
||||
select 'b' as id, 20 as counter, '2016-01-01T00:00:00Z'::timestamp as timestamp_col
|
||||
|
||||
{% elif var('version') == 2 %}
|
||||
|
||||
select 'a' as id, 30 as counter, '2016-01-02T00:00:00Z'::timestamp as timestamp_col union all
|
||||
select 'b' as id, 20 as counter, '2016-01-01T00:00:00Z'::timestamp as timestamp_col union all
|
||||
select 'c' as id, 40 as counter, '2016-01-02T00:00:00Z'::timestamp as timestamp_col
|
||||
|
||||
{% else %}
|
||||
|
||||
select 'a' as id, 30 as counter, '2016-01-02T00:00:00Z'::timestamp as timestamp_col union all
|
||||
select 'c' as id, 40 as counter, '2016-01-02T00:00:00Z'::timestamp as timestamp_col
|
||||
|
||||
{% endif %}
|
||||
|
||||
{% endsnapshot %}
|
||||
"""
|
||||
|
||||
expected_csv = """
|
||||
id,counter,timestamp_col,dbt_scd_id,dbt_updated_at,dbt_valid_from,dbt_valid_to
|
||||
a,10,2016-01-01 00:00:00.000,927354aa091feffd9437ead0bdae7ae1,2016-07-01 00:00:00.000,2016-07-01 00:00:00.000,2016-07-02 00:00:00.000
|
||||
b,20,2016-01-01 00:00:00.000,40ace4cbf8629f1720ec8a529ed76f8c,2016-07-01 00:00:00.000,2016-07-01 00:00:00.000,
|
||||
a,30,2016-01-02 00:00:00.000,e9133f2b302c50e36f43e770944cec9b,2016-07-02 00:00:00.000,2016-07-02 00:00:00.000,
|
||||
c,40,2016-01-02 00:00:00.000,09d33d35101e788c152f65d0530b6837,2016-07-02 00:00:00.000,2016-07-02 00:00:00.000,
|
||||
""".lstrip()
|
||||
|
||||
|
||||
@pytest.fixture(scope="class")
|
||||
def snapshots():
|
||||
return {"snapshot_check_cols_updated_at_actual.sql": snapshot_sql}
|
||||
|
||||
|
||||
@pytest.fixture(scope="class")
|
||||
def seeds():
|
||||
return {"snapshot_check_cols_updated_at_expected.csv": expected_csv}
|
||||
|
||||
|
||||
@pytest.fixture(scope="class")
|
||||
def project_config_update():
|
||||
return {
|
||||
"seeds": {
|
||||
"quote_columns": False,
|
||||
"test": {
|
||||
"snapshot_check_cols_updated_at_expected": {
|
||||
"+column_types": {
|
||||
"timestamp_col": "timestamp without time zone",
|
||||
"dbt_updated_at": "timestamp without time zone",
|
||||
"dbt_valid_from": "timestamp without time zone",
|
||||
"dbt_valid_to": "timestamp without time zone",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def test_simple_snapshot(project):
|
||||
"""
|
||||
Test that the `dbt_updated_at` column reflects the `updated_at` timestamp expression in the config.
|
||||
|
||||
Approach:
|
||||
1. Create a table that represents the expected data after a series of snapshots
|
||||
- Use dbt seed to create the expected relation (`snapshot_check_cols_updated_at_expected`)
|
||||
2. Execute a series of snapshots to create the data
|
||||
- Use a series of (3) dbt snapshot commands to create the actual relation (`snapshot_check_cols_updated_at_actual`)
|
||||
- The logic can switch between 3 different versions of the data (depending on the `version` number)
|
||||
- The `updated_at` value is passed in via `--vars` and cast to a timestamp in the snapshot config
|
||||
3. Compare the two relations for equality
|
||||
"""
|
||||
|
||||
# 1. Create a table that represents the expected data after a series of snapshots
|
||||
results = run_dbt(["seed", "--show", "--vars", "{version: 1, updated_at: 2016-07-01}"])
|
||||
assert len(results) == 1
|
||||
|
||||
# 2. Execute a series of snapshots to create the data
|
||||
|
||||
# Snapshot day 1
|
||||
results = run_dbt(["snapshot", "--vars", "{version: 1, updated_at: 2016-07-01}"])
|
||||
assert len(results) == 1
|
||||
|
||||
# Snapshot day 2
|
||||
results = run_dbt(["snapshot", "--vars", "{version: 2, updated_at: 2016-07-02}"])
|
||||
assert len(results) == 1
|
||||
|
||||
# Snapshot day 3
|
||||
results = run_dbt(["snapshot", "--vars", "{version: 3, updated_at: 2016-07-03}"])
|
||||
assert len(results) == 1
|
||||
|
||||
# 3. Compare the two relations for equality
|
||||
check_relations_equal(
|
||||
project.adapter,
|
||||
["snapshot_check_cols_updated_at_actual", "snapshot_check_cols_updated_at_expected"],
|
||||
compare_snapshot_cols=True,
|
||||
)
|
||||
Reference in New Issue
Block a user