Compare commits

...

104 Commits

Author SHA1 Message Date
Jeremy Cohen
2c8856da3b Add changelog entry 2023-07-19 12:37:50 +02:00
Jeremy Cohen
029045e556 Pin sqlparse<0.5 2023-07-19 12:35:31 +02:00
Jeremy Cohen
433e5c670e Pin click<9 2023-07-19 12:35:17 +02:00
FishtownBuildBot
cbfc6a8baf Add new index.html and changelog yaml files from dbt-docs (#8131) 2023-07-19 11:23:35 +02:00
Michelle Ark
9765596247 wrap deprecation warnings in warn_or_error calls (#8129) 2023-07-18 18:08:52 -04:00
Kshitij Aranke
1b1a291fae Publish coverage results to codecov.io (#8127) 2023-07-18 16:47:44 -05:00
FishtownBuildBot
867534c1f4 Cleanup main after cutting new 1.6.latest branch (#8102) 2023-07-17 19:49:55 -04:00
Michelle Ark
6d8b6459eb bumpversion 1.7.0a1 (#8111) 2023-07-17 18:10:29 -04:00
Mike Alfare
203bd8defd Apply new integration tests to existing framework to identify supported features (#8099)
* applied new integration tests to existing framework

* applied new integration tests to existing framework

* generalized tests for reusability in adapters; fixed drop index issue

* generalized tests for reusability in adapters; fixed drop index issue

* removed unnecessary overrides in tests

* adjusted import to allow for usage in adapters

* adjusted import to allow for usage in adapters

* removed fixture artifact

* generalized the materialized view fixture which will need to be specific to the adapter

* unskipped tests in the test runner package

* corrected test condition

* corrected test condition

* added missing initial build for the relation type swap tests
2023-07-17 12:17:02 -04:00
Emily Rockman
949680a5ce add env vars for datadog ci visibility (#8097)
* add env vars for datadog ci visibility

* modify pytest command for tracing

* fix posargs

* move env vars to job that needs them

* add test repeater to DD

* swap flags
2023-07-17 09:52:21 -05:00
Damian Owsianny
015c490b63 Fix query comment tests (#7928) (#7928) 2023-07-13 13:45:14 -06:00
Quigley Malcolm
95a916936e [CT-2821] Support dbt-semantic-interfaces~=0.1.0rc1 (#8085)
* Bump version support for `dbt-semantic-interfaces` to `~=0.1.0rc1`

* Add tests for asserting WhereFilter satisfies protocol

* Add `call_parameter_sets` to `WhereFilter` class to satisfy protocol

* Changie doc for moving to DSI 0.1.0rc1

* [CT-2822]  Fix `NonAdditiveDimension` Implementation (#8089)

* Add test to ensure `NonAdditiveDimension` implementation satisfies protocol

* Fix typo in `NonAdditiveDimension`: `window_grouples` -> `window_groupings`

* Add changie doc for typo fix in NonAdditiveDimension
2023-07-13 12:51:23 +02:00
Michelle Ark
961d69d8c2 gitignore user.yml and profiles.yml (#8087) 2023-07-12 17:46:49 -07:00
Michelle Ark
be4d0a5b88 add __test__ = False to non-test classes that start with Test (#8086) 2023-07-12 17:35:38 -07:00
Quigley Malcolm
5310d3715c CT-2691 Fix the populating of a Metric's depends_on property (#8015)
* Add metrics from metric type params to a metric's depends_on

* Add Lookup utility for finding `SemanticModel`s by measure names

* Add the `SemanticModel` of a `Metric`'s measure property to the `Metric`'s `depends_on`

* Add `SemanticModelConfig` to `SemanticModel`

Some tests were failing due to `Metric`'s referencing `SemanticModels`.
Specifically there was a check to see if a referenced node was disabled,
and because `SemanticModel`'s didn't have a `config` holding the `enabled`
boolean attr, core would blow up.

* Checkpoint on test fixing

* Correct metricflow_time_spine_sql in test fixtures

* Add check for `SemanticModel` nodes in `Linker.link_node`

Now that `Metrics` depend on `SemanticModels` and `SemanticModels`
have their own dependencies on `Models` they need to be checked for
in the `Linker.link_node`. I forget the details but things blow up
without it. Basically it adds the SemanticModels to the dependency
graph.

* Fix artifacts/test_previous_version_state.py tests

* fix access/test_access.py tests

* Fix function metric tests

* Fix functional partial_parsing tests

* Add time dimension to semantic model in exposures fixture

* Bump DSI version to a minimum of 0.1.0dev10

DSI 0.1.0dev10 fixes an incoherence issue in DSI around `agg_time_dimension`
setting. This incoherence was that `measure.agg_time_dimension` was being
required, even though it was no longer supposed to be a required attribute
(it's specificially typed as optional in the protocol). This was causing
a handful of tests to fail because the `semantic_model.defaults.agg_time_dimension`
value wasn't being respected. Pulling in the fix from DSI 0.1.0dev10 fixes
the issue.

Interestingly after bumping the DSI version, the integration tests were
still failing. If I ran the tests individually they passed though. To get
`make integration` to run properly I ended up having to clear my `.tox`
cache, as it seems some outdated state was being persisted.

* Add test specifically for checking the `depends_on` of `Metric` nodes

* Re-enable test asserting calling metric nodes in models

* Migrate `checked_agg_time_dimension` to `checked_agg_time_dimension_for_measure`

DSI 0.1.0dev10 moved `checked_agg_time_dimension` from the `Measure`
protocol to the `SemanticModel` protocol as `checked_agg_time_dimension_for_measure`.
This finishes a change where for a given measure either the `Measure.agg_time_dimension`
or the measure's parent `SemanticModel.defaults.agg_time_dimension` needs to be
set, instead of always require the measure's `Measure.agg_time_dimension`.

* Add changie doc for populating metric

---------

Co-authored-by: Gerda Shank <gerda@dbtlabs.com>
2023-07-12 13:42:44 -07:00
Jeremy Cohen
6bdf983e0b Add semantic_models to tracked resource counts (#8078)
* Add semantic_models to tracked resource counts

* Add changelog entry

* Simplify node statistic tabulation.

* Remove review comment. Replace with explanation.

---------

Co-authored-by: Peter Allen Webb <peter.webb@dbtlabs.com>
2023-07-12 14:19:19 -04:00
Michelle Ark
6604b9ca31 8030/fix contract checksum (#8072) 2023-07-12 09:36:15 -07:00
Emily Rockman
305241fe86 Er/ct 2675 test custom target (#8079)
* remove skip

* fix retry test
2023-07-12 11:03:19 -05:00
Michelle Ark
2d686b73fd update contributing.md reference to test/integration (#8073) 2023-07-12 09:03:02 -07:00
Alex Rosenfeld
30def98ed9 Remove volume declaration (#8069)
* Remove volume declaration

* Changelog entry

---------

Co-authored-by: Doug Beatty <doug.beatty@dbtlabs.com>
2023-07-12 10:39:24 -04:00
Thomas Lento
b78d23f68d Update validate sql test classes to new nomenclature (#8076)
The original implementation of validate_sql was called dry_run,
but in the rename the test classes and much of their associated
documentation still retained the old naming.

This is mainly cosmetic, but since these test classes will be
imported into adapter repositories we should fix this now before
the wrong name proliferates.
2023-07-12 10:20:25 +02:00
Thomas Lento
4ffd633e40 Add validate_sql method to base adapter with implementation for SQLAdapters (#8001)
* Add dry_run method to base adapter with implementation for SQLAdapters

resolves #7839

In the CLI integration, MetricFlow will issue dry run queries as
part of its warehouse-level validation of the semantic manifest,
including all semantic model and metric definitions.

In most cases, issuing an `explain` query is adequate, however,
BigQuery does not support the `explain` keyword and so we cannot
simply pre-pend `explain` to our input queries and expect the
correct behavior across all contexts.

This commit adds a dry_run() method to the BaseAdapter which mirrors
the execute() method in that it simply delegates to the ConnectionManager.
It also adds a working implementation to the SQLConnectionManager and
includes a few test cases for adapter maintainers to try out on their own.

The current implementation should work out of the box with most
of our adapters. BigQuery will require us to implement the dry_run
method on the BigQueryConnectionManager, and community-maintained
adapters can opt in by enabling the test and ensuring their own
implementations work as expected.

Note - we decided to make these concrete methods that throw runtime
exceptions for direct descendants of BaseAdapter in order to avoid
forcing community adapter maintainers to implement a method that does
not currently have any use cases in dbt proper.

* Switch dry_run implementation to be macro-based

The common pattern for engine-specific SQL statement construction
in dbt is to provide a default macro which can then be overridden
on a per-adapter basis by either adapter maintainers or end users.
The advantage of this is users can take advantage of alternative
SQL syntax for performance or other reasons, or even to enable
local usage if an engine relies on a non-standard expression and
the adapter maintainer has not updated the package.

Although there are some risks here they are minimal, and the benefit
of added expressiveness and consistency with other similar constructs
is clear, so we adopt this approach here.

* Improve error message for InvalidConnectionError in test_invalid_dry_run.

* Rename dry_run to validate_sql

The validate_sql name has less chance of colliding with dbt's
command nomenclature, both now and in some future where we have
dry-run operations.

* Rename macro and test files to validate_sql

* Fix changelog entry
2023-07-11 18:24:18 -04:00
Kshitij Aranke
07c3dcd21c Fixes #7785: fail-fast behavior (#8066) 2023-07-11 17:05:35 -05:00
Doug Beatty
fd233eac62 Use Ubuntu 22.04.2 LTS (Jammy Jellyfish) since it is a long-term supported release (#8071)
* Use Ubuntu 22.04.2 LTS (Jammy Jellyfish) since it is a long-term supported release

* Changelog entry
2023-07-11 15:56:31 -06:00
Emily Rockman
d8f38ca48b Flaky Test Workflow (#8055)
* add permissions

* replace db setup

* try with bash instead of just pytest flags

* fix test command

* remove spaces

* remove force-flaky flag

* add starting vlaues

* add mac and windows postgres isntall

* define use bash

* fix typo

* update output report

* tweak last if condition

* clarify failures/successful runs

* print running success and failure tally

* just output pytest instead of capturing it

* set shell to not exit immediately on exit code

* add formatting around results for easier scanning

* more output formatting

* add matrix to unlock parallel runners

* increase to ten batches

* update debug

* add comment

* clean up comments
2023-07-11 12:58:46 -05:00
Quigley Malcolm
7740bd6b45 Remove create_metric as a public facing SemanticModel.Measure property (#8068)
* Remove `create_metric` as a public facing `SemanticModel.Measure` property

We want to add `create_metric`. The `create_metric` property will be
incredibly useful. However, at this time it is not hooked up, and we don't
have time to hook it up before the code freeze for 1.6.0rc of core. As
it doesn't do anything, we shouldn't allow people to specify it, because
it won't do what one would expect. We plan on making the implementation
of `create_metric` a priority for 1.7 of core

* Changie doc for the removal of create_metric property
2023-07-11 09:36:10 -07:00
dave-connors-3
a57fdf008e add negative part number test case for split part cross db util (#7200)
* add negative test case

* changie

* missed a comma

* Update changelog entry

* Add a negative number (rather than subtract a positive number)

---------

Co-authored-by: Doug Beatty <44704949+dbeatty10@users.noreply.github.com>
Co-authored-by: Doug Beatty <doug.beatty@dbtlabs.com>
2023-07-11 08:56:52 -06:00
Peter Webb
a8e3afe8af Fix accidental propagation of log messages to root logger (#7882)
* Fix accidental propagation of log messages to root logger.

* Add changelog entry

* Fixed an issue which blocked debug logging to stdout with --log-level debug, unless --debug was also used.
2023-07-11 10:40:33 -04:00
Peter Webb
44572e72f0 Semantic Model Validation (#8049)
* Use dbt-semantic-interface validations on semantic models and metrics defined in Core.

* Remove empty test, since semantic models don't generate any validation warnings.

* Add changelog entry.

* Temporarily remove requirement that there must be semantic models definied in order to define metrics
2023-07-10 14:48:20 -04:00
Nathaniel May
54b1e5699c Update the PR template (#7892)
* add interface changes section to the PR template

* update entire template

* split up choices for tests and interfaces

* minor formatting change

* add line breaks

* actually put in line breaks

* revert split choices in checklist

* add line breaks to top

* move docs link

* typo
2023-07-10 13:13:02 -04:00
Chenyu Li
ee7bc24903 partial parse file path (#8032) 2023-07-10 08:52:52 -07:00
Emily Rockman
15ef88d2ed add workflow for flaky test testing (#8044)
* add workflow for flaky test testing

* improve docs

* rename workflow

* update default input

* add min passing tests
2023-07-07 12:48:40 -05:00
Emily Rockman
7c56d72b46 pin click (#8050)
* pin click

* changelog
2023-07-07 11:20:27 -05:00
Michelle Ark
5d28e4744e ModelNodeArgs.unique_id - include v (#8038) 2023-07-06 11:54:12 -04:00
Jeremy Cohen
746ca7d149 Nicer error message for contracted model missing 'columns' (#8024) 2023-07-06 11:06:56 +02:00
Grant Murray
a58b5ee8fb CT-2780 [Docs] Fix-toc-links-in-contributing-md (#8017)
* docs(contributing): fix-toc-link-in-contributing-md

* docs(contributing-md): fix-link2

* Fix-typo

* Remove backtick from href

* changie new

* Cough commit / trigger CI
2023-07-05 11:47:43 +02:00
Peter Webb
7fbfd53c3e CT-2752: Extract methods to new SemanticManifest class for better encapsulation of details. (#8012) 2023-07-01 16:59:44 -04:00
Chenyu Li
4c44c29ee4 fire proper event for inline query error (#7960) 2023-06-30 14:50:52 -07:00
FishtownBuildBot
8ee0fe0a64 [Automated] Merged prep-release/1.6.0b8_5425945126 into target main during release process 2023-06-30 14:18:53 -05:00
Github Build Bot
307a618ea8 Bumping version to 1.6.0b8 and generate changelog 2023-06-30 18:36:40 +00:00
Michelle Ark
ce07ce58e1 versioned node selection with underscore delimiting (#7995) 2023-06-30 14:28:19 -04:00
Michelle Ark
7ea51df6ae allow on_schema_change: fail for incremental models with contracts (#8006) 2023-06-30 13:27:48 -04:00
Gerda Shank
fe463c79fe Add time spine table configuration to semantic manifest (#7996)
* Bump dbt-semantic-interface to dev8

* Create time_spline_table_configuration in semantic manifest

* Add metricflow_time_spin to semantic_models tests

* Remove skip from test

* Changie

* Update exception message

Co-authored-by: Quigley Malcolm <QMalcolm@users.noreply.github.com>

---------

Co-authored-by: Quigley Malcolm <QMalcolm@users.noreply.github.com>
2023-06-30 13:22:24 -04:00
d-kaneshiro
d7d6843c5f Added note before running integration tests (#7657)
Co-authored-by: Kshitij Aranke <kshitij.aranke@dbtlabs.com>
2023-06-30 09:32:08 -05:00
Tyler Rouze
adcf8bcbb3 [CT-2551] Make state selection MECE (#7773)
* ct-2551: adds old and unmodified state selection methods

* ct-2551: update check_unmodified_content to simplify

* add unit and integration tests for unmodified and old

* add changelog entry

* ct-2551: reformatting of contingent adapter assignment list
2023-06-30 09:30:10 -04:00
Gerda Shank
5d937802f1 Remove pin of sqlparse, minor refactoring, add tests (#7993) 2023-06-29 16:24:17 -04:00
Michelle Ark
8c201e88a7 type + fix typo in ModelNodeArgs.unique_id (#7992) 2023-06-29 13:10:26 -04:00
d-kaneshiro
b8bc264731 Unified to UTC (#7665)
* UnifiedToUTC

* Check proximity of dbt_valid_to and deleted time

* update the message to print if the assertion fails

* add CHANGELOG entries

* test only if naive

* Added comments about naive and aware

* Generalize comparison of datetimes that are "close enough"

---------

Co-authored-by: Doug Beatty <doug.beatty@dbtlabs.com>
2023-06-29 11:52:25 -04:00
Quigley Malcolm
9c6fbff0c3 CT-2707: Populate metric input measures (#7984)
* Fix tests fixtures which were using measures for metric numerator/denominators

In our previous upgrade to DSI dev7, numerators and denominators for
metrics switched from being `MetricInputMeasure`s to `MetricInput`s.
I.e. metric numerators and denominators should references other metrics,
not semantic model measures. However, at that time, we weren't actually
doing anything with numerators and denominators in core, so no issue
got raised. The changes we are about to make though are going to surface
these issues..

* Add tests for ensuring a metric's `input_measures` gets properly populated

* Begin populating `metric.type_params.input_measures`

This isn't my favorite bit of code. Mostly because there are checks for
existence which really should be handled before this point, however a
good point for that to happen doesn't exist currently. For instance,
in an ideal world by the time we get to `_process_metric_node`, if a
metric is of type `RATIO` and the nominator and denominator should be
guaranteed.

* Update test checking that disabled metrics aren't added to the manifest metrics

We updated from the metric `number_of_people` to `average_tenure_minus_people` for
this test because disabling `number_of_people` raised other exceptions at parse
time due to a metric referencing a disabled metric. The metric `average_tenure_minus_people`
is a leaf metric, and so for this test, it is a better candidate.

* Update `test_disabled_metric_ref_model` to have more disabled metrics

There are metrics which depend on the metric `number_of_people`. If
`number_of_people` is disabled without the metrics that depend on it
being disabled, then a different (expected) exception would be raised
than the one this test is testing for. Thus we've disabled those
downstream metrics.

* Add test which checks that metrics depending on disabled metrics raise an exception

* Add changie doc for populating metric input measures
2023-06-29 08:30:58 -07:00
Kshitij Aranke
5c7aa7f9ce dbt clone (#7881)
Co-authored-by: Matthew McKnight <matthew.mcknight@dbtlabs.com>
2023-06-28 19:22:07 -05:00
Peter Webb
1af94dedad CT-2757: Fix unit test which broke due to merge issues (#7978) 2023-06-28 17:34:45 -04:00
Gerda Shank
2e7c968419 Use events.contextvar because of multiprocessing unable to pickle ContextVar (#7949)
* Add task contextvars to events/contextvars.py

* Use events.contextvars instead of task.contextvars

* Changie
2023-06-28 16:55:50 -04:00
Jeremy Cohen
05b0820a9e Replace space with underscore in NodeType strings (#7947) 2023-06-28 20:04:32 +02:00
FishtownBuildBot
d4e620eb50 [Automated] Merged prep-release/1.6.0b7_5402737814 into target main during release process 2023-06-28 11:03:18 -05:00
Will Bryant
0f52505dbe Fix CTE insertion position when the model uses WITH RECURSIVE (#7350) (#7414) 2023-06-28 11:41:31 -04:00
Github Build Bot
cb754fd97b Bumping version to 1.6.0b7 and generate changelog 2023-06-28 15:11:49 +00:00
Michelle Ark
e01d4c0a6e Add restrict-access to dbt_project.yml (#7962) 2023-06-28 10:55:11 -04:00
Michelle Ark
7a6bedaae3 consolidate cross-project ref entrypoint + plugin framework (#7955) 2023-06-28 10:54:55 -04:00
Niall Woodward
22145e7e5f Add invocation command flag (#7939)
* Add invocation command flag

* Add changie entry

* Update .changes/unreleased/Features-20230623-111254.yaml
2023-06-28 10:47:07 -04:00
Niall Woodward
b3ac41ff9a Add thread_id context var (#7942)
* Add thread_id context var

* Changie

* Fix context test

* Update .changes/unreleased/Features-20230623-173357.yaml

Co-authored-by: Doug Beatty <44704949+dbeatty10@users.noreply.github.com>

* Fix tests

---------

Co-authored-by: Doug Beatty <44704949+dbeatty10@users.noreply.github.com>
2023-06-28 10:44:51 -04:00
Michelle Ark
036b95e5b2 Handle state:modified for external nodes (#7925) 2023-06-28 10:34:17 -04:00
Rainer Mensing
2ce0c5ccf5 Add merge incremental strategy for postgres (#6951)
* Add merge incremental strategy

* Expect merge to be a valid strategy for Postgres

---------

Co-authored-by: Anders Swanson <anders.swanson@dbtlabs.com>
Co-authored-by: Doug Beatty <doug.beatty@dbtlabs.com>
2023-06-28 10:28:40 -04:00
Peter Webb
7156cc5c1d Add Partial Parsing Support for Semantic Models (#7964)
* CT-2711: Add partial parsing support for semantic models

* CT-2711: Fix typo identified in code review
2023-06-27 17:40:27 -04:00
Michelle Ark
fcd30b1de2 Set access model node args (#7966) 2023-06-27 16:10:13 -04:00
Michelle Ark
a84fa50166 allow setting enabled and depends_on_nodes from ModelNodeArgs (#7930) 2023-06-27 16:09:35 -04:00
Doug Beatty
6a1e3a6db8 Fix macro namespace search packages (#5804) 2023-06-27 14:55:38 -04:00
Peter Webb
b37e5b5198 Factor Out Repeated Logic in the PartialParsing Class (#7952)
* CT-2711: Add remove_tests() call to delete_schema_source() so that call sites are more uniform with other node deletion call sites. This will enable further code factorization.

* CT-2711: Factor repeated code section (mostly) out of PartialParsing.handle_schema_file_changes()

* CT-2711: Factor a repeated code section out of schedule_nodes_for_parsing()
2023-06-26 15:20:50 -04:00
Doug Beatty
f9d4e9e03d Fix comment for dbt retry (#7932) 2023-06-26 12:10:44 -06:00
Gerda Shank
9c97d30702 update mashumaro to 3.8.1 (#7951)
* Update mashumaro to 3.8

* Change to 3.8.1

* Changie
2023-06-26 14:01:22 -04:00
FishtownBuildBot
9836f7bdef [Automated] Merged prep-release/1.6.0b6_5360267609 into target main during release process 2023-06-23 16:11:51 -05:00
Github Build Bot
b07ff7aebd Bumping version to 1.6.0b6 and generate changelog 2023-06-23 20:31:02 +00:00
Peter Webb
aecbb4564c CT-2732: Fix selector methods to include semantic models (#7936) 2023-06-23 15:54:33 -04:00
Quigley Malcolm
779663b39c Improved Semantic Model Measure Percentile defaults (#7877)
* Update semantic model parsing test to check measure agg params

* Make `use_discrete_percentile` and `use_approximate_percentile` non optional and default false

This was a mistake in our implementation of the MeasureAggregationParams.
We had defined them as optional and defaulting to `None`. However, as the
protocol states, they cannot be `None`, they must be a boolean value.
Thus now we now ensure them.

* Add changie doc for measure percentile fixes
2023-06-23 11:20:59 -07:00
Quigley Malcolm
7934af2974 Improved Semantic expr specification handling (#7876)
* Update semantic model parsing test to check different measure expr types

* Allow semantic model measure exprs to be defined with ints and bools in yaml

Sometimes the expr for a measure can defined in yaml with a bool or an int.
However, we were only allowing for strings. There was a work around for this,
which was wrapping your bool or int in double quotes in the yaml, but
this can be fairly annoying for the end user.

* Changie doc for fixing measure expr yaml specification
2023-06-23 10:34:11 -07:00
FishtownBuildBot
533988233e [Automated] Merged prep-release/1.6.0b5_5349512955 into target main during release process 2023-06-22 15:07:48 -05:00
Peter Webb
8bc0e77a1d CT-2719: Rename the semantic_nodes collection on the manifest to semantic_models (#7927) 2023-06-22 15:59:54 -04:00
Github Build Bot
1c93c9bb58 Bumping version to 1.6.0b5 and generate changelog 2023-06-22 19:27:29 +00:00
Michelle Ark
6d7b32977c Fix: safe remove of external nodes from nodes.depends_on (#7923) 2023-06-22 13:42:06 -04:00
Michelle Ark
bf15466bec UninstalledPackagesFoundError references correct packages specified path (#7886) 2023-06-22 11:40:59 -04:00
Gerda Shank
fb1ebe48f0 Resolve SemanticModel refs in the same way as other refs (#7895) 2023-06-22 11:26:08 -04:00
Peter Webb
de65697ff9 Further Integrate Semantic Models (#7917)
* CT-2651: Add Semantic Models to the manifest and various pieces of graph linking code

* CT-2651: Finish integrating semantic models into the partial parsing system

* CT-2651: More semantic model details for partial parsing

* CT-2651: Remove merged references to project_dependencies

* CT-2651: Revise changelog entry

* CT-2651: Disable unit test until partial parsing of semantic models is complete.

* CT-2651: Temporarily disable an apparently-flaky test.
2023-06-21 18:31:54 -04:00
Michelle Ark
ecf90d689e Refactor/unify public and model nodes (#7891) 2023-06-21 17:12:15 -04:00
Mila Page
4cdeff11cd Remove --config-dir instead of fixing #7774 (#7793)
* Stringfy the dir always to solve the bug.

* Add changelog

---------

Co-authored-by: Mila Page <versusfacit@users.noreply.github.com>
2023-06-21 11:36:47 -07:00
Gerda Shank
9ff2f6e430 Do not jinja render packages from dependencies yml (#7910)
* Add some comments to methods constructing Project/RuntimeConfig

* Save flag that packages dict came from dependencies.yml

* Test for not rendering packages_dict

* Changie

* Ensure packages_yml_dict and dependencies_yml_dict are dictionaries

* Ensure "packages" passed to render_packages is a dict
2023-06-21 14:33:23 -04:00
Jeremy Cohen
73a0dc6d14 Reorganize, annotate, and revise dependency pins (#7368)
* Reorganize + annotate dependencies

* Loosen pins on other dbt-labs packages

* Minor pins, no patch pins

* Rm support for py37

* Fix protobuf pin

* Bump networkx upper bound #6551
2023-06-21 07:08:43 +02:00
Kshitij Aranke
0a1c73e0fd Fixes #7753: Fix regression in run-operation to not require the name of the package to run (#7811) 2023-06-20 11:19:16 -05:00
Quigley Malcolm
8653ffc5a4 Upgrade core to support dbt-semantic-interfaces 0.1.0dev7 (#7903)
* Bump DSI dependency version to 0.1.0dev7

* Cleaner DSI type enum importing

Previoulsy we had to use individual import paths for each type enum
that dbt-semantic-interfaces provided. However, dbt-semantic-interfaces
has been updated to allow for importing all the type enums from a
singular path.

* Cleaner DSI protocol importing

Previoulsy we had to use individual import paths for each protocol
that dbt-semantic-interfaces provided. However, dbt-semantic-interfaces
has been updated to allow for importing all the protocols from a
singular path.

* Add semantic protocol satisifcation test for metric type params

* Replace `metric.type_params.measures` with `metric.type_params.input_measures`

In DSI 0.1.0dev7 `measures` on metric type params became `input_measures`.
Additionally `input_measures` should not be user specified but something
we compile at parse time, thus we've removed it from `UnparsedMetricTypeParams`.
Finally, actually populating `input_measures` is somewhat complicated due
to the existance of derived metrics, thus that work is being pushed
off to CT-2707.

* Update metric numerator/denominator to be `MetricInput`s

In DSI 0.1.0dev7 `metric.type_params.numerator` and `metric.type_params.denominator`
switched from being `MetricInputMeasure`s to `MetricInput`s. This
commit reflects that change. Additionally, some helper functions on
metric type params were removed related to the numerator and denominator.
Thus we've removed them respectively in this commit.

* Add protocol satisfaction tests for `MetricInput` and `MetricInputMeasure`

* Add `post_aggregation_reference` to `MetricInput` and fix typo in `MetricInputMeasure`

DSI 0.1.0dev7 added `post_aggregation_reference` to the `MetricInput` protocol,
thus we've added it to our implementation in core. Additionally, we had a typo
in a method name in our implementation of `MetricInputMeasure`, ironically
a similar function to the one we've added for `MetricInput`

* Changie doc for upgraded to DSI 0.1.0dev7

* Fix parsing of metric numerator and denominator in schema_yaml_readers

Previously numerator and denominator of a metric were `MetricInputMeasure`s,
now they're `MetricInput`s. Changing the typing isn't enough though.
We have parsing functions in `schema_yaml_readers` which were specifically
parsing the numerator and denominator as if they were `MetricInputMeasure`s.
Thus we had to updating the schema_yaml_readers to parse them as `MetricInput`s.
During this we had some logic in a parsing function `_get_metric_inputs` which
could be abstracted to newly added functions.
2023-06-20 08:08:57 -07:00
Quigley Malcolm
86583a350f Ct-2690 Support dbt-semantic-interfaces 0.1.0dev5 (#7888)
* Upgrade to dbt-semantic-interfaces v0.1.0dev5

This is a fairly simple upgrade. Literally it's just pointing at the
the new versions. The v3 schemas are directly compatible with v5 because
there were no protocol level changes from v3 to v5. All the changers were
updates to tools MetricFlow uses from DSI, not tools that we ourselves
are using in core (yet).

* Add changie doc for DSI version bump
2023-06-16 12:47:55 -07:00
Quigley Malcolm
fafab5d557 [CT-2696] Skip jinia parsing of metric filters (#7885)
* Update metric filters in testing fixtures

I incorrectly wrote the tests such that they didn't include curly
braces, `{{..}}`, around things like `dimension(..)` for filters.
This updates the tests fixtures to have proper filter specifications

* Skip jinja rendering of `filter` key of metrics

Note that `filter` can show up in multiple places: as a root key
on a metric (`metric.filter`), on a metric input (`metric.type_params.metrics[x].filter`),
denominator (`metric.type_params.denominator.filter`), numerator
(`metric.type_params.numerator.filter`), and a metric input measure
(`metric.type_params.measure.filter` and `metric.type_params.measures[x].filter`).
In this commit we skip all of them :)

* Add changie doc for skipping jinja parsing for metric filters

* Update yaml renderer test for metrics
2023-06-15 15:41:31 -07:00
Gerda Shank
39e0c22353 Allow setting packages in dependencies.yml and move dependencies to runtime config (#7857) 2023-06-15 14:41:57 -04:00
colin-rogers-dbt
f767943fb2 Add AdapterRegistered event log message (#7862)
* Add AdapterRegistered event log message

* Add AdapterRegistered to unit test

* make versioning and logging consistent

* make versioning and logging consistent

* add to_version_string

* remove extra equals

* format fire_event
2023-06-14 14:56:47 -07:00
dependabot[bot]
ae97831ebf Bump mypy from 0.981 to 1.0.1 (#7027)
* Bump mypy from 0.981 to 1.0.1

Bumps [mypy](https://github.com/python/mypy) from 0.981 to 1.0.1.
- [Release notes](https://github.com/python/mypy/releases)
- [Commits](https://github.com/python/mypy/compare/v0.981...v1.0.1)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

* upgrade mypy and fix all errors

* fixing some duplicate imports from conflict resolution

* fix mypy errors from merging in main

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com>
2023-06-14 14:31:57 -05:00
Jeremy Cohen
f16bae0ab9 Fix: dbt show --inline with private models (#7838)
* Add functional test

* Check resource_type before DbtReferenceError

* Changelog entry
2023-06-14 12:23:20 -04:00
FishtownBuildBot
b947b2bc7e [Automated] Merged prep-release/1.6.0b4_5260249343 into target main during release process 2023-06-13 16:40:37 -05:00
Github Build Bot
7068688181 Bumping version to 1.6.0b4 and generate changelog 2023-06-13 20:54:08 +00:00
Quigley Malcolm
38c0600982 Update SemanticModel node to match DSI 0.1.0dev3 protocols (#7848)
* Add tests to ensure our semantic layer nodes satisfy the DSI protocols

These tests create runtime checkable versions of the protocols defined in
DSI. Thus we can instantiate instances of our semantic layer nodes and
use `isinstance` to check that they satisfy the protocol. These `runtime_checkable`
versions of the protocols should only exist in testing and should never
be used in the actual package code.

* Update the `Dimension` object of `SemanticModel` node to match DSI protocol

* Make `UnparsedDimension` more strict and update schema readers accordingly

* Update the `Entity` object of `SemanticModel` node to match DSI protocol

* Make `UnparsedEntity` more strict and update schema readers accordingly

* Update the `Measure` object of `SemanticModel` node to match DSI protocol

* Make `UnparsedMeasure` more strict and update schema readers accordingly

* Update the `SemanticModel` node to match DSI protocol

A lot of the additions are helper functions which we don't actually
use in core. This is a known issue. We're in the process of removing
a fair number of them from the DSI protocol spec. However, in the meantime
we need to implement them to satisfy the protocol unfortunately.

* Make `UnparsedSemanticModel` more strict and update schema readers accordingly

* Changie entry for updating SemanticModel node
2023-06-13 13:25:35 -07:00
Jeremy Cohen
83d163add5 Respect column quote config in model contracts (#7537) 2023-06-13 15:32:56 -04:00
Emily Rockman
d46e8855ef Allow ProjectDependency to have extra fields (#7834)
* Allow ProjectDependency to have extra fields

* changelog
2023-06-13 09:47:50 -05:00
mirnawong1
60524c0f8e update adapters url (#7779)
* update adapters url

in response to [docs.getedbt.com pr 3465](https://github.com/dbt-labs/docs.getdbt.com/issues/3465), updating this error message to point to the correct URL, which was recently changed. 

old URL: https://docs.getdbt.com/docs/supported-data-platforms#adapter-installation
new URL: https://docs.getdbt.com/docs/connect-adapters#install-using-the-cli

thank you @dbeatty10 for your 🦅 👀 !

* adding changie entry

* Update .changes/unreleased/Breaking Changes-20230612-161159.yaml

---------

Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com>
2023-06-13 08:51:21 -05:00
Gerda Shank
ca73a2aa15 Use project directory in path selector instead of cwd (#7829)
* Use contextvar to store and get project_root for path selector method

* Changie

* Modify test to check Path selector with project-dir

* Don't set cv_project_root in base task if no config
2023-06-13 09:08:09 -04:00
Emily Rockman
4a833a4272 Move remaining unit tests to /tests directory (#7843)
* moved remaining unit tests to  directory

* fix path

* Delete profiles.yml

* rename remaining test refs
2023-06-13 07:08:28 -05:00
FishtownBuildBot
f9abeca231 Add new index.html and changelog yaml files from dbt-docs (#7836) 2023-06-12 11:55:42 -04:00
Jeremy Cohen
5f9e527768 Rm spaces from NodeType strings (#7842)
* Rm space from NodeType strings

* Add changelog entry
2023-06-12 11:55:07 -04:00
FishtownBuildBot
6f51de4cb5 [Automated] Merged prep-release/1.6.0b3_5215324721 into target main during release process 2023-06-08 15:52:09 -05:00
305 changed files with 7074 additions and 4434 deletions

View File

@@ -1,5 +1,5 @@
[bumpversion]
current_version = 1.6.0b3
current_version = 1.7.0a1
parse = (?P<major>[\d]+) # major version number
\.(?P<minor>[\d]+) # minor version number
\.(?P<patch>[\d]+) # patch version number

View File

@@ -3,6 +3,7 @@
For information on prior major and minor releases, see their changelogs:
* [1.6](https://github.com/dbt-labs/dbt-core/blob/1.6.latest/CHANGELOG.md)
* [1.5](https://github.com/dbt-labs/dbt-core/blob/1.5.latest/CHANGELOG.md)
* [1.4](https://github.com/dbt-labs/dbt-core/blob/1.4.latest/CHANGELOG.md)
* [1.3](https://github.com/dbt-labs/dbt-core/blob/1.3.latest/CHANGELOG.md)

View File

@@ -1 +0,0 @@
## dbt-core 1.6.0-a1 - April 17, 2023

View File

@@ -1,50 +0,0 @@
## dbt-core 1.6.0-b1 - May 12, 2023
### Features
- Skip catalog generation ([#6980](https://github.com/dbt-labs/dbt-core/issues/6980))
- Publication artifacts and cross-project ref ([#7227](https://github.com/dbt-labs/dbt-core/issues/7227))
- Add graph structure summaries to target path output ([#7357](https://github.com/dbt-labs/dbt-core/issues/7357))
- Allow duplicate manifest node (models, seeds, analyses, snapshots) names across packages ([#7446](https://github.com/dbt-labs/dbt-core/issues/7446))
- Detect breaking changes to enforced constraints ([#7065](https://github.com/dbt-labs/dbt-core/issues/7065))
- Check for project dependency cycles ([#7468](https://github.com/dbt-labs/dbt-core/issues/7468))
### Fixes
- Persist timing info in run results for failed nodes ([#5476](https://github.com/dbt-labs/dbt-core/issues/5476))
- fix typo in unpacking statically parsed ref ([#7364](https://github.com/dbt-labs/dbt-core/issues/7364))
- safe version attribute access in _check_resource_uniqueness ([#7375](https://github.com/dbt-labs/dbt-core/issues/7375))
- Fix dbt command missing target-path param ([# 7411](https://github.com/dbt-labs/dbt-core/issues/ 7411))
- Fix v0 ref resolution ([#7408](https://github.com/dbt-labs/dbt-core/issues/7408))
- Add --target-path to dbt snapshot command. ([#7418](https://github.com/dbt-labs/dbt-core/issues/7418))
- dbt build selection of tests' descendants ([#7289](https://github.com/dbt-labs/dbt-core/issues/7289))
- fix groupable node partial parsing, raise DbtReferenceError at runtime for safety ([#7437](https://github.com/dbt-labs/dbt-core/issues/7437))
- Fix partial parsing of latest_version changes for downstream references ([#7369](https://github.com/dbt-labs/dbt-core/issues/7369))
- Use "add_node" to update depends_on.nodes ([#7453](https://github.com/dbt-labs/dbt-core/issues/7453))
- Fix var precedence in configs: root vars override package vars ([#6705](https://github.com/dbt-labs/dbt-core/issues/6705))
- Fix inverted `--print/--no-print` flag ([#7517](https://github.com/dbt-labs/dbt-core/issues/7517))
- Back-compat for previous return type of 'collect_freshness' macro ([#7489](https://github.com/dbt-labs/dbt-core/issues/7489))
- print model version in dbt show if specified ([#7407](https://github.com/dbt-labs/dbt-core/issues/7407))
- enable dbt show for seeds ([#7273](https://github.com/dbt-labs/dbt-core/issues/7273))
- push down limit filtering to adapter ([#7390](https://github.com/dbt-labs/dbt-core/issues/7390))
- Allow missing `profiles.yml` for `dbt deps` and `dbt init` ([#7511](https://github.com/dbt-labs/dbt-core/issues/7511))
- `run_results.json` is now written after every node completes. ([#7302](https://github.com/dbt-labs/dbt-core/issues/7302))
- Do not rewrite manifest.json during 'docs serve' command ([#7553](https://github.com/dbt-labs/dbt-core/issues/7553))
- Pin protobuf to greater than 4.0.0 ([#7565](https://github.com/dbt-labs/dbt-core/issues/7565))
- inject sql header in query for show ([#7413](https://github.com/dbt-labs/dbt-core/issues/7413))
- Pin urllib3 to ~=1.0 ([#7573](https://github.com/dbt-labs/dbt-core/issues/7573))
- Throw error for duplicated versioned and unversioned models ([#7487](https://github.com/dbt-labs/dbt-core/issues/7487))
### Under the Hood
- Update docs link in ContractBreakingChangeError message ([#7366](https://github.com/dbt-labs/dbt-core/issues/7366))
- Reduce memory footprint of cached statement results. ([#7281](https://github.com/dbt-labs/dbt-core/issues/7281))
- Remove noisy parsing events: GenericTestFileParse, MacroFileParse, Note events for static model parsing ([#6671](https://github.com/dbt-labs/dbt-core/issues/6671))
- Update --help text for cache-related parameters ([#7381](https://github.com/dbt-labs/dbt-core/issues/7381))
- Small UX improvements to model versions: Support defining latest_version in unsuffixed file by default. Notify on unpinned ref when a prerelease version is available. ([#7443](https://github.com/dbt-labs/dbt-core/issues/7443))
### Contributors
- [@AndyBys](https://github.com/AndyBys) ([#6980](https://github.com/dbt-labs/dbt-core/issues/6980))
- [@b-luu](https://github.com/b-luu) ([#7289](https://github.com/dbt-labs/dbt-core/issues/7289))
- [@dwreeves](https://github.com/dwreeves) ([#7418](https://github.com/dbt-labs/dbt-core/issues/7418))
- [@thomasgjerdekog](https://github.com/thomasgjerdekog) ([#7517](https://github.com/dbt-labs/dbt-core/issues/7517))

View File

@@ -1,38 +0,0 @@
## dbt-core 1.6.0-b2 - May 25, 2023
### Features
- Added warnings for model and ref deprecations ([#7433](https://github.com/dbt-labs/dbt-core/issues/7433))
- Update drop_relation macro to allow for configuration of drop statement separately from object name ([#7625](https://github.com/dbt-labs/dbt-core/issues/7625))
- accept publications in dbt.invoke ([#7372](https://github.com/dbt-labs/dbt-core/issues/7372))
### Fixes
- Honor `--skip-profile-setup` parameter when inside an existing project ([#7594](https://github.com/dbt-labs/dbt-core/issues/7594))
- Fix: Relative project paths weren't working with deps ([#7491](https://github.com/dbt-labs/dbt-core/issues/7491))
- Exclude password fields from Jinja rendering. ([#7629](https://github.com/dbt-labs/dbt-core/issues/7629))
- Add --target-path to more CLI subcommands ([#7646](https://github.com/dbt-labs/dbt-core/issues/7646))
- Stringify flag paths for Jinja context ([#7495](https://github.com/dbt-labs/dbt-core/issues/7495))
- write run_results.json for run operation ([#7502](https://github.com/dbt-labs/dbt-core/issues/7502))
### Under the Hood
- Add ability to instantiate Flags class from dict ([#7607](https://github.com/dbt-labs/dbt-core/issues/7607))
- Add other relation to reffable nodes ([#7550](https://github.com/dbt-labs/dbt-core/issues/7550))
- Move node patch method to schema parser patch_node_properties and refactor schema parsing ([#7430](https://github.com/dbt-labs/dbt-core/issues/7430))
- Remove legacy file logger code ([#NA](https://github.com/dbt-labs/dbt-core/issues/NA))
- Break up integration tests as a short term fix for Windows CI runs ([#7668](https://github.com/dbt-labs/dbt-core/issues/7668))
- Include null checks in utils test base ([#7670](https://github.com/dbt-labs/dbt-core/issues/7670))
- Write pub artifact to log ([#7372](https://github.com/dbt-labs/dbt-core/issues/7372))
### Dependencies
- Bump ubuntu from 23.04 to 23.10 ([#7675](https://github.com/dbt-labs/dbt-core/pull/7675))
- ([#7681](https://github.com/dbt-labs/dbt-core/pull/7681))
### Contributors
- [@dradnan89@hotmail.com](https://github.com/dradnan89@hotmail.com) ([#7681](https://github.com/dbt-labs/dbt-core/pull/7681))
- [@dwreeves](https://github.com/dwreeves) ([#7646](https://github.com/dbt-labs/dbt-core/issues/7646))
- [@iknox-fa](https://github.com/iknox-fa) ([#7491](https://github.com/dbt-labs/dbt-core/issues/7491), [#NA](https://github.com/dbt-labs/dbt-core/issues/NA))
- [@sdebruyn](https://github.com/sdebruyn) ([#7670](https://github.com/dbt-labs/dbt-core/issues/7670))
- [@stu-k](https://github.com/stu-k) ([#7607](https://github.com/dbt-labs/dbt-core/issues/7607), [#7550](https://github.com/dbt-labs/dbt-core/issues/7550))

View File

@@ -1,46 +0,0 @@
## dbt-core 1.6.0-b3 - June 08, 2023
### Breaking Changes
- Dropped support for Python 3.7 ([#7082](https://github.com/dbt-labs/dbt-core/issues/7082))
- Switch from dbt-metrics to dbt-semantic-interfaces for MetricNode definitions ([#7500](https://github.com/dbt-labs/dbt-core/issues/7500), [#7404](https://github.com/dbt-labs/dbt-core/issues/7404))
### Features
- Add support for materialized views ([#6911](https://github.com/dbt-labs/dbt-core/issues/6911))
- Optimize template rendering for common parse scenarios ([#7449](https://github.com/dbt-labs/dbt-core/issues/7449))
- nodes in packages respect custom generate_alias_name, generate_schema_name, generate_database_name macro overrides defined in packages ([#7444](https://github.com/dbt-labs/dbt-core/issues/7444))
- Enable state for deferral to be separate from state for selectors ([#7300](https://github.com/dbt-labs/dbt-core/issues/7300))
- add access selection syntax ([#7738](https://github.com/dbt-labs/dbt-core/issues/7738))
- add project_name to manifest metadata ([#7752](https://github.com/dbt-labs/dbt-core/issues/7752))
- dbt retry ([#7299](https://github.com/dbt-labs/dbt-core/issues/7299))
- Revamp debug, add --connection flag. Prepare for future refactors/interface changes. ([#7104](https://github.com/dbt-labs/dbt-core/issues/7104))
- Validate public models are not materialized as ephemeral ([#7226](https://github.com/dbt-labs/dbt-core/issues/7226))
- Added support for parsing and serializaing semantic models ([#7499](https://github.com/dbt-labs/dbt-core/issues/7499), [#7503](https://github.com/dbt-labs/dbt-core/issues/7503))
### Fixes
- Constraint rendering fixes: wrap check expression in parentheses, foreign key 'references', support expression in all constraint types ([#7417](https://github.com/dbt-labs/dbt-core/issues/7417), [#7480](https://github.com/dbt-labs/dbt-core/issues/7480), [#7416](https://github.com/dbt-labs/dbt-core/issues/7416))
- Fix warning messages for deprecated dbt_project.yml configs ([#7424](https://github.com/dbt-labs/dbt-core/issues/7424))
- Add `%` to adapter suite test cases for `persist_docs` ([#7698](https://github.com/dbt-labs/dbt-core/issues/7698))
- Improve warnings for constraints and materialization types ([#7335](https://github.com/dbt-labs/dbt-core/issues/7335))
- Incorrect paths used for "target" and "state" directories ([#7465](https://github.com/dbt-labs/dbt-core/issues/7465))
- fix StopIteration error when publication for project not found ([#7711](https://github.com/dbt-labs/dbt-core/issues/7711))
- Using version 0 works when resolving single model ([#7372](https://github.com/dbt-labs/dbt-core/issues/7372))
- Fix empty --warn-error-options error message ([#7730](https://github.com/dbt-labs/dbt-core/issues/7730))
- send sql header on contract enforcement ([#7714](https://github.com/dbt-labs/dbt-core/issues/7714))
- Fixed doc link in selector.py ([#7533](https://github.com/dbt-labs/dbt-core/issues/7533))
- Fix null-safe equals comparison via `equals` ([#7778](https://github.com/dbt-labs/dbt-core/issues/7778))
- Log PublicationArtifactAvailable even when partially parsing unchanged public models ([#7782](https://github.com/dbt-labs/dbt-core/issues/7782))
- fix RuntimeError when removing project dependency from dependencies.yml ([#7743](https://github.com/dbt-labs/dbt-core/issues/7743))
### Under the Hood
- Fix flaky test for --fail-fast ([#7744](https://github.com/dbt-labs/dbt-core/issues/7744))
- Create `add_from_artifact` to populate `state_relation` field of nodes ([#7551](https://github.com/dbt-labs/dbt-core/issues/7551))
### Contributors
- [@dave-connors-3](https://github.com/dave-connors-3) ([#7738](https://github.com/dbt-labs/dbt-core/issues/7738))
- [@quazi-irfan](https://github.com/quazi-irfan) ([#7533](https://github.com/dbt-labs/dbt-core/issues/7533))
- [@sdebruyn](https://github.com/sdebruyn) ([#7082](https://github.com/dbt-labs/dbt-core/issues/7082))
- [@stu-k](https://github.com/stu-k) ([#7299](https://github.com/dbt-labs/dbt-core/issues/7299), [#7551](https://github.com/dbt-labs/dbt-core/issues/7551))

View File

@@ -1,6 +0,0 @@
kind: Breaking Changes
body: Dropped support for Python 3.7
time: 2023-05-15T05:31:48.375649+02:00
custom:
Author: sdebruyn
Issue: "7082"

View File

@@ -1,6 +0,0 @@
kind: Breaking Changes
body: Switch from dbt-metrics to dbt-semantic-interfaces for MetricNode definitions
time: 2023-06-07T19:03:09.680189-07:00
custom:
Author: QMalcolm
Issue: 7500 7404

View File

@@ -1,6 +0,0 @@
kind: "Dependencies"
body: "Bump ubuntu from 23.04 to 23.10"
time: 2023-05-22T00:59:48.00000Z
custom:
Author: dependabot[bot]
PR: 7675

View File

@@ -1,5 +0,0 @@
kind: Dependencies
time: 2023-05-22T21:22:01.336704+02:00
custom:
Author: dradnan89@hotmail.com
PR: "7681"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Skip catalog generation
time: 2023-03-21T21:33:38.513443Z
custom:
Author: AndyBys
Issue: "6980"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Add support for materialized views
time: 2023-03-29T12:03:13.862041-04:00
custom:
Author: mikealfare McKnight-42
Issue: "6911"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Publication artifacts and cross-project ref
time: 2023-04-20T12:47:56.92683-04:00
custom:
Author: gshank
Issue: "7227"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Optimize template rendering for common parse scenarios
time: 2023-04-24T16:36:11.24088-04:00
custom:
Author: peterallenwebb
Issue: "7449"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Add graph structure summaries to target path output
time: 2023-04-25T14:25:22.269051-04:00
custom:
Author: peterallenwebb
Issue: "7357"

View File

@@ -1,7 +0,0 @@
kind: Features
body: Allow duplicate manifest node (models, seeds, analyses, snapshots) names across
packages
time: 2023-04-29T15:50:57.757832-04:00
custom:
Author: MichelleArk
Issue: "7446"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Detect breaking changes to enforced constraints
time: 2023-05-03T10:11:00.617936-05:00
custom:
Author: emmyoop
Issue: "7065"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Check for project dependency cycles
time: 2023-05-09T09:41:47.2-04:00
custom:
Author: gshank
Issue: "7468"

View File

@@ -1,7 +0,0 @@
kind: Features
body: nodes in packages respect custom generate_alias_name, generate_schema_name,
generate_database_name macro overrides defined in packages
time: 2023-05-09T21:29:35.557761-04:00
custom:
Author: michelleark
Issue: "7444"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Added warnings for model and ref deprecations
time: 2023-05-09T23:33:29.679333-04:00
custom:
Author: peterallenwebb
Issue: "7433"

View File

@@ -1,7 +0,0 @@
kind: Features
body: Update drop_relation macro to allow for configuration of drop statement separately
from object name
time: 2023-05-15T12:23:04.177141-04:00
custom:
Author: mikealfare
Issue: "7625"

View File

@@ -1,6 +0,0 @@
kind: Features
body: accept publications in dbt.invoke
time: 2023-05-17T18:56:27.585784-04:00
custom:
Author: MichelleArk
Issue: "7372"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Enable state for deferral to be separate from state for selectors
time: 2023-05-23T22:59:55.920975-07:00
custom:
Author: aranke
Issue: "7300"

View File

@@ -1,6 +0,0 @@
kind: Features
body: add access selection syntax
time: 2023-05-30T16:48:47.740037-05:00
custom:
Author: dave-connors-3
Issue: "7738"

View File

@@ -1,6 +0,0 @@
kind: Features
body: add project_name to manifest metadata
time: 2023-06-01T13:22:23.259448-04:00
custom:
Author: michelleark jtcohen6
Issue: "7752"

View File

@@ -1,6 +0,0 @@
kind: Features
body: dbt retry
time: 2023-06-02T08:33:02.410456-07:00
custom:
Author: stu-k aranke
Issue: "7299"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Revamp debug, add --connection flag. Prepare for future refactors/interface changes.
time: 2023-06-04T02:59:56.28283-07:00
custom:
Author: versusfacit
Issue: 7104

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Fix null-safe equals comparison via `equals`
time: 2023-06-04T08:00:52.537967-06:00
custom:
Author: dbeatty10
Issue: "7778"

View File

@@ -1,6 +0,0 @@
kind: Features
body: ' Validate public models are not materialized as ephemeral'
time: 2023-06-05T22:20:39.382019-04:00
custom:
Author: michelleark
Issue: "7226"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Added support for parsing and serializaing semantic models
time: 2023-06-06T16:53:51.117429-04:00
custom:
Author: peterallenwebb
Issue: 7499 7503

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Persist timing info in run results for failed nodes
time: 2023-04-13T13:31:57.938847-05:00
custom:
Author: stu-k
Issue: "5476"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: fix typo in unpacking statically parsed ref
time: 2023-04-14T16:36:42.279838-04:00
custom:
Author: MichelleArk
Issue: "7364"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: safe version attribute access in _check_resource_uniqueness
time: 2023-04-18T13:52:57.367108-04:00
custom:
Author: MichelleArk
Issue: "7375"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Fix dbt command missing target-path param
time: 2023-04-19T14:21:50.959786-07:00
custom:
Author: ChenyuLInx
Issue: "\t7411"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Fix v0 ref resolution
time: 2023-04-19T22:09:10.155137-04:00
custom:
Author: MichelleArk
Issue: "7408"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Add --target-path to dbt snapshot command.
time: 2023-04-20T10:42:54.17972-04:00
custom:
Author: dwreeves
Issue: "7418"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: dbt build selection of tests' descendants
time: 2023-04-21T17:24:28.335866975+02:00
custom:
Author: b-luu
Issue: "7289"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: fix groupable node partial parsing, raise DbtReferenceError at runtime for safety
time: 2023-04-24T16:18:43.130637-04:00
custom:
Author: MichelleArk
Issue: "7437"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Fix partial parsing of latest_version changes for downstream references
time: 2023-04-24T16:46:49.721231-04:00
custom:
Author: MichelleArk
Issue: "7369"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Use "add_node" to update depends_on.nodes
time: 2023-04-24T17:34:04.37479-04:00
custom:
Author: gshank
Issue: "7453"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: 'Fix var precedence in configs: root vars override package vars'
time: 2023-04-27T23:07:14.992529-04:00
custom:
Author: MichelleArk
Issue: "6705"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: 'Constraint rendering fixes: wrap check expression in parentheses, foreign key
''references'', support expression in all constraint types'
time: 2023-05-04T14:06:42.545193-04:00
custom:
Author: MichelleArk
Issue: 7417 7480 7416

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Fix inverted `--print/--no-print` flag
time: 2023-05-05T13:25:45.949997-06:00
custom:
Author: dbeatty10 thomasgjerdekog
Issue: "7517"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Back-compat for previous return type of 'collect_freshness' macro
time: 2023-05-06T17:33:15.104953+02:00
custom:
Author: jtcohen6
Issue: "7489"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Fix warning messages for deprecated dbt_project.yml configs
time: 2023-05-06T18:09:00.361961+02:00
custom:
Author: jtcohen6
Issue: "7424"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: print model version in dbt show if specified
time: 2023-05-08T04:27:07.9965-07:00
custom:
Author: aranke
Issue: "7407"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: enable dbt show for seeds
time: 2023-05-08T04:49:22.82093-07:00
custom:
Author: aranke
Issue: "7273"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: push down limit filtering to adapter
time: 2023-05-08T06:09:26.455524-07:00
custom:
Author: aranke
Issue: "7390"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Allow missing `profiles.yml` for `dbt deps` and `dbt init`
time: 2023-05-08T07:29:29.873793-06:00
custom:
Author: dbeatty10
Issue: "7511"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: '`run_results.json` is now written after every node completes.'
time: 2023-05-08T09:37:32.809356-05:00
custom:
Author: iknox-fa
Issue: "7302"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Do not rewrite manifest.json during 'docs serve' command
time: 2023-05-08T14:25:18.376379-04:00
custom:
Author: jtcohen6
Issue: "7553"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: inject sql header in query for show
time: 2023-05-09T10:29:32.12947-07:00
custom:
Author: aranke
Issue: "7413"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Pin protobuf to greater than 4.0.0
time: 2023-05-09T12:31:59.019718-04:00
custom:
Author: gshank
Issue: "7565"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Pin urllib3 to ~=1.0
time: 2023-05-09T15:35:30.504674-04:00
custom:
Author: mikealfare
Issue: "7573"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Throw error for duplicated versioned and unversioned models
time: 2023-05-09T16:50:07.530882-04:00
custom:
Author: gshank
Issue: "7487"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Honor `--skip-profile-setup` parameter when inside an existing project
time: 2023-05-11T14:04:41.382181-06:00
custom:
Author: dbeatty10
Issue: "7594"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: 'Fix: Relative project paths weren''t working with deps'
time: 2023-05-15T12:36:54.807413-05:00
custom:
Author: iknox-fa
Issue: "7491"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Exclude password fields from Jinja rendering.
time: 2023-05-15T14:28:51.400321-04:00
custom:
Author: peterallenwebb
Issue: "7629"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Add --target-path to more CLI subcommands
time: 2023-05-16T15:26:44.557072-04:00
custom:
Author: dwreeves
Issue: "7646"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: write run_results.json for run operation
time: 2023-05-22T13:29:24.182612-07:00
custom:
Author: aranke
Issue: "7502"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Stringify flag paths for Jinja context
time: 2023-05-22T13:50:07.897354-04:00
custom:
Author: gshank
Issue: "7495"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Add `%` to adapter suite test cases for `persist_docs`
time: 2023-05-24T16:06:48.477708-06:00
custom:
Author: dbeatty10
Issue: "7698"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Improve warnings for constraints and materialization types
time: 2023-05-25T07:36:51.855641-05:00
custom:
Author: emmyoop
Issue: "7335"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Incorrect paths used for "target" and "state" directories
time: 2023-05-25T16:50:53.718564-04:00
custom:
Author: gshank
Issue: "7465"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: fix StopIteration error when publication for project not found
time: 2023-05-26T15:37:38.952939-04:00
custom:
Author: michelleark
Issue: "7711"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Using version 0 works when resolving single model
time: 2023-05-26T16:47:27.6065-04:00
custom:
Author: gshank
Issue: "7372"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Fix empty --warn-error-options error message
time: 2023-05-30T10:42:28.382804-04:00
custom:
Author: michelleark
Issue: "7730"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: send sql header on contract enforcement
time: 2023-05-31T13:19:19.801391-04:00
custom:
Author: michelleark
Issue: "7714"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Fixed doc link in selector.py
time: 2023-06-01T13:05:49.952-05:00
custom:
Author: quazi-irfan
Issue: "7533"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Log PublicationArtifactAvailable even when partially parsing unchanged public
models
time: 2023-06-05T12:11:27.739183-04:00
custom:
Author: michelleark
Issue: "7782"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: fix RuntimeError when removing project dependency from dependencies.yml
time: 2023-06-05T12:44:25.978022-04:00
custom:
Author: michelleark
Issue: "7743"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Update docs link in ContractBreakingChangeError message
time: 2023-04-17T11:45:01.005104+02:00
custom:
Author: jtcohen6
Issue: "7366"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Reduce memory footprint of cached statement results.
time: 2023-04-17T12:27:21.972268-05:00
custom:
Author: iknox-fa
Issue: "7281"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: 'Remove noisy parsing events: GenericTestFileParse, MacroFileParse, Note events
for static model parsing'
time: 2023-04-18T12:12:36.928665+02:00
custom:
Author: jtcohen6
Issue: "6671"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Update --help text for cache-related parameters
time: 2023-04-18T12:23:23.276693+02:00
custom:
Author: jtcohen6
Issue: "7381"

View File

@@ -1,8 +0,0 @@
kind: Under the Hood
body: 'Small UX improvements to model versions: Support defining latest_version in
unsuffixed file by default. Notify on unpinned ref when a prerelease version is
available. '
time: 2023-04-24T13:53:00.484916+02:00
custom:
Author: jtcohen6
Issue: "7443"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Add ability to instantiate Flags class from dict
time: 2023-05-15T09:51:16.291352-05:00
custom:
Author: stu-k
Issue: "7607"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Add other relation to reffable nodes
time: 2023-05-15T15:21:07.808892-05:00
custom:
Author: stu-k
Issue: "7550"

View File

@@ -1,7 +0,0 @@
kind: Under the Hood
body: Move node patch method to schema parser patch_node_properties and refactor schema
parsing
time: 2023-05-16T09:42:41.793503-04:00
custom:
Author: gshank
Issue: "7430"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Remove legacy file logger code
time: 2023-05-18T11:42:51.825799-05:00
custom:
Author: iknox-fa
Issue: NA

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Break up integration tests as a short term fix for Windows CI runs
time: 2023-05-19T15:30:59.138043-04:00
custom:
Author: mikealfare
Issue: "7668"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Include null checks in utils test base
time: 2023-05-21T12:57:20.659726+02:00
custom:
Author: sdebruyn
Issue: "7670"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Write pub artifact to log
time: 2023-05-23T12:21:37.235938-04:00
custom:
Author: gshank
Issue: "7372"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Fix flaky test for --fail-fast
time: 2023-06-02T14:57:43.266412-04:00
custom:
Author: michelleark
Issue: "7744"

View File

@@ -1,6 +0,0 @@
kind: Under the Hood
body: Create `add_from_artifact` to populate `state_relation` field of nodes
time: 2023-06-05T23:47:06.581326-07:00
custom:
Author: stu-k aranke
Issue: "7551"

View File

@@ -0,0 +1,6 @@
kind: Dependencies
body: Pin click<9 + sqlparse<0.5
time: 2023-07-19T12:37:43.716495+02:00
custom:
Author: jtcohen6
PR: "8146"

View File

@@ -0,0 +1,6 @@
kind: Docs
body: Fix for column tests not rendering on quoted columns
time: 2023-05-31T11:54:19.687363-04:00
custom:
Author: drewbanin
Issue: "201"

View File

@@ -0,0 +1,6 @@
kind: Docs
body: Remove static SQL codeblock for metrics
time: 2023-07-18T19:24:22.155323+02:00
custom:
Author: marcodamore
Issue: "436"

View File

@@ -0,0 +1,6 @@
kind: Fixes
body: Enable converting deprecation warnings to errors
time: 2023-07-18T12:55:18.03914-04:00
custom:
Author: michelleark
Issue: "8130"

View File

@@ -1,23 +1,35 @@
resolves #
resolves #
[docs](https://github.com/dbt-labs/docs.getdbt.com/issues/new/choose) dbt-labs/docs.getdbt.com/#
<!---
Include the number of the issue addressed by this PR above if applicable.
PRs for code changes without an associated issue *will not be merged*.
See CONTRIBUTING.md for more information.
Include the number of the docs issue that was opened for this PR. If
this change has no user-facing implications, "N/A" suffices instead. New
docs tickets can be created by clicking the link above or by going to
https://github.com/dbt-labs/docs.getdbt.com/issues/new/choose.
-->
### Description
### Problem
<!---
Describe the Pull Request here. Add any references and info to help reviewers
understand your changes. Include any tradeoffs you considered.
Describe the problem this PR is solving. What is the application state
before this PR is merged?
-->
### Solution
<!---
Describe the way this PR solves the above problem. Add as much detail as you
can to help reviewers understand your changes. Include any alternatives and
tradeoffs you considered.
-->
### Checklist
- [ ] I have read [the contributing guide](https://github.com/dbt-labs/dbt-core/blob/main/CONTRIBUTING.md) and understand what's expected of me
- [ ] I have signed the [CLA](https://docs.getdbt.com/docs/contributor-license-agreements)
- [ ] I have run this code in development and it appears to resolve the stated issue
- [ ] I have read [the contributing guide](https://github.com/dbt-labs/dbt-core/blob/main/CONTRIBUTING.md) and understand what's expected of me
- [ ] I have run this code in development and it appears to resolve the stated issue
- [ ] This PR includes tests, or tests are not required/relevant for this PR
- [ ] I have [opened an issue to add/update docs](https://github.com/dbt-labs/docs.getdbt.com/issues/new/choose), or docs changes are not required/relevant for this PR
- [ ] I have run `changie new` to [create a changelog entry](https://github.com/dbt-labs/dbt-core/blob/main/CONTRIBUTING.md#adding-a-changelog-entry)
- [ ] This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX

View File

@@ -73,7 +73,6 @@ jobs:
env:
TOXENV: "unit"
PYTEST_ADDOPTS: "-v --color=yes --csv unit_results.csv"
steps:
- name: Check out the repository
@@ -101,17 +100,17 @@ jobs:
CURRENT_DATE=$(date +'%Y-%m-%dT%H_%M_%S') # no colons allowed for artifacts
echo "date=$CURRENT_DATE" >> $GITHUB_OUTPUT
- uses: actions/upload-artifact@v3
if: always()
with:
name: unit_results_${{ matrix.python-version }}-${{ steps.date.outputs.date }}.csv
path: unit_results.csv
- name: Upload Unit Test Coverage to Codecov
if: ${{ matrix.python-version == '3.11' }}
uses: codecov/codecov-action@v3
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
integration:
name: integration test / python ${{ matrix.python-version }} / ${{ matrix.os }}
runs-on: ${{ matrix.os }}
timeout-minutes: 45
timeout-minutes: 60
strategy:
fail-fast: false
@@ -126,11 +125,15 @@ jobs:
env:
TOXENV: integration
PYTEST_ADDOPTS: "-v --color=yes -n4 --csv integration_results.csv"
DBT_INVOCATION_ENV: github-actions
DBT_TEST_USER_1: dbt_test_user_1
DBT_TEST_USER_2: dbt_test_user_2
DBT_TEST_USER_3: dbt_test_user_3
DD_CIVISIBILITY_AGENTLESS_ENABLED: true
DD_API_KEY: ${{ secrets.DATADOG_API_KEY }}
DD_SITE: datadoghq.com
DD_ENV: ci
DD_SERVICE: ${{ github.event.repository.name }}
steps:
- name: Check out the repository
@@ -161,7 +164,7 @@ jobs:
tox --version
- name: Run tests
run: tox
run: tox -- --ddtrace
- name: Get current date
if: always()
@@ -176,11 +179,11 @@ jobs:
name: logs_${{ matrix.python-version }}_${{ matrix.os }}_${{ steps.date.outputs.date }}
path: ./logs
- uses: actions/upload-artifact@v3
if: always()
with:
name: integration_results_${{ matrix.python-version }}_${{ matrix.os }}_${{ steps.date.outputs.date }}.csv
path: integration_results.csv
- name: Upload Integration Test Coverage to Codecov
if: ${{ matrix.python-version == '3.11' }}
uses: codecov/codecov-action@v3
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
build:
name: build packages

155
.github/workflows/test-repeater.yml vendored Normal file
View File

@@ -0,0 +1,155 @@
# **what?**
# This workflow will test all test(s) at the input path given number of times to determine if it's flaky or not. You can test with any supported OS/Python combination.
# This is batched in 10 to allow more test iterations faster.
# **why?**
# Testing if a test is flaky and if a previously flaky test has been fixed. This allows easy testing on supported python versions and OS combinations.
# **when?**
# This is triggered manually from dbt-core.
name: Flaky Tester
on:
workflow_dispatch:
inputs:
branch:
description: 'Branch to check out'
type: string
required: true
default: 'main'
test_path:
description: 'Path to single test to run (ex: tests/functional/retry/test_retry.py::TestRetry::test_fail_fast)'
type: string
required: true
default: 'tests/functional/...'
python_version:
description: 'Version of Python to Test Against'
type: choice
options:
- '3.8'
- '3.9'
- '3.10'
- '3.11'
os:
description: 'OS to run test in'
type: choice
options:
- 'ubuntu-latest'
- 'macos-latest'
- 'windows-latest'
num_runs_per_batch:
description: 'Max number of times to run the test per batch. We always run 10 batches.'
type: number
required: true
default: '50'
permissions: read-all
defaults:
run:
shell: bash
jobs:
debug:
runs-on: ubuntu-latest
steps:
- name: "[DEBUG] Output Inputs"
run: |
echo "Branch: ${{ inputs.branch }}"
echo "test_path: ${{ inputs.test_path }}"
echo "python_version: ${{ inputs.python_version }}"
echo "os: ${{ inputs.os }}"
echo "num_runs_per_batch: ${{ inputs.num_runs_per_batch }}"
pytest:
runs-on: ${{ inputs.os }}
strategy:
# run all batches, even if one fails. This informs how flaky the test may be.
fail-fast: false
# using a matrix to speed up the jobs since the matrix will run in parallel when runners are available
matrix:
batch: ["1", "2", "3", "4", "5", "6", "7", "8", "9", "10"]
env:
PYTEST_ADDOPTS: "-v --color=yes -n4 --csv integration_results.csv"
DBT_TEST_USER_1: dbt_test_user_1
DBT_TEST_USER_2: dbt_test_user_2
DBT_TEST_USER_3: dbt_test_user_3
DD_CIVISIBILITY_AGENTLESS_ENABLED: true
DD_API_KEY: ${{ secrets.DATADOG_API_KEY }}
DD_SITE: datadoghq.com
DD_ENV: ci
DD_SERVICE: ${{ github.event.repository.name }}
steps:
- name: "Checkout code"
uses: actions/checkout@v3
with:
ref: ${{ inputs.branch }}
- name: "Setup Python"
uses: actions/setup-python@v4
with:
python-version: "${{ inputs.python_version }}"
- name: "Setup Dev Environment"
run: make dev
- name: "Set up postgres (linux)"
if: inputs.os == 'ubuntu-latest'
run: make setup-db
# mac and windows don't use make due to limitations with docker with those runners in GitHub
- name: "Set up postgres (macos)"
if: inputs.os == 'macos-latest'
uses: ./.github/actions/setup-postgres-macos
- name: "Set up postgres (windows)"
if: inputs.os == 'windows-latest'
uses: ./.github/actions/setup-postgres-windows
- name: "Test Command"
id: command
run: |
test_command="python -m pytest ${{ inputs.test_path }}"
echo "test_command=$test_command" >> $GITHUB_OUTPUT
- name: "Run test ${{ inputs.num_runs_per_batch }} times"
id: pytest
run: |
set +e
for ((i=1; i<=${{ inputs.num_runs_per_batch }}; i++))
do
echo "Running pytest iteration $i..."
python -m pytest --ddtrace ${{ inputs.test_path }}
exit_code=$?
if [[ $exit_code -eq 0 ]]; then
success=$((success + 1))
echo "Iteration $i: Success"
else
failure=$((failure + 1))
echo "Iteration $i: Failure"
fi
echo
echo "==========================="
echo "Successful runs: $success"
echo "Failed runs: $failure"
echo "==========================="
echo
done
echo "failure=$failure" >> $GITHUB_OUTPUT
- name: "Success and Failure Summary: ${{ inputs.os }}/Python ${{ inputs.python_version }}"
run: |
echo "Batch: ${{ matrix.batch }}"
echo "Successful runs: ${{ steps.pytest.outputs.success }}"
echo "Failed runs: ${{ steps.pytest.outputs.failure }}"
- name: "Error for Failures"
if: ${{ steps.pytest.outputs.failure }}
run: |
echo "Batch ${{ matrix.batch }} failed ${{ steps.pytest.outputs.failure }} of ${{ inputs.num_runs_per_batch }} tests"
exit 1

2
.gitignore vendored
View File

@@ -29,6 +29,8 @@ var/
.mypy_cache/
.dmypy.json
logs/
.user.yml
profiles.yml
# PyInstaller
# Usually these files are written by a python script from a template

View File

@@ -37,7 +37,7 @@ repos:
alias: flake8-check
stages: [manual]
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v0.981
rev: v1.3.0
hooks:
- id: mypy
# N.B.: Mypy is... a bit fragile.

View File

@@ -5,151 +5,12 @@
- "Breaking changes" listed under a version may require action from end users or external maintainers when upgrading to that version.
- Do not edit this file directly. This file is auto-generated using [changie](https://github.com/miniscruff/changie). For details on how to document a change, see [the contributing guide](https://github.com/dbt-labs/dbt-core/blob/main/CONTRIBUTING.md#adding-changelog-entry)
## dbt-core 1.6.0-b3 - June 08, 2023
### Breaking Changes
- Dropped support for Python 3.7 ([#7082](https://github.com/dbt-labs/dbt-core/issues/7082))
- Switch from dbt-metrics to dbt-semantic-interfaces for MetricNode definitions ([#7500](https://github.com/dbt-labs/dbt-core/issues/7500), [#7404](https://github.com/dbt-labs/dbt-core/issues/7404))
### Features
- Add support for materialized views ([#6911](https://github.com/dbt-labs/dbt-core/issues/6911))
- Optimize template rendering for common parse scenarios ([#7449](https://github.com/dbt-labs/dbt-core/issues/7449))
- nodes in packages respect custom generate_alias_name, generate_schema_name, generate_database_name macro overrides defined in packages ([#7444](https://github.com/dbt-labs/dbt-core/issues/7444))
- Enable state for deferral to be separate from state for selectors ([#7300](https://github.com/dbt-labs/dbt-core/issues/7300))
- add access selection syntax ([#7738](https://github.com/dbt-labs/dbt-core/issues/7738))
- add project_name to manifest metadata ([#7752](https://github.com/dbt-labs/dbt-core/issues/7752))
- dbt retry ([#7299](https://github.com/dbt-labs/dbt-core/issues/7299))
- Revamp debug, add --connection flag. Prepare for future refactors/interface changes. ([#7104](https://github.com/dbt-labs/dbt-core/issues/7104))
- Validate public models are not materialized as ephemeral ([#7226](https://github.com/dbt-labs/dbt-core/issues/7226))
- Added support for parsing and serializaing semantic models ([#7499](https://github.com/dbt-labs/dbt-core/issues/7499), [#7503](https://github.com/dbt-labs/dbt-core/issues/7503))
### Fixes
- Constraint rendering fixes: wrap check expression in parentheses, foreign key 'references', support expression in all constraint types ([#7417](https://github.com/dbt-labs/dbt-core/issues/7417), [#7480](https://github.com/dbt-labs/dbt-core/issues/7480), [#7416](https://github.com/dbt-labs/dbt-core/issues/7416))
- Fix warning messages for deprecated dbt_project.yml configs ([#7424](https://github.com/dbt-labs/dbt-core/issues/7424))
- Add `%` to adapter suite test cases for `persist_docs` ([#7698](https://github.com/dbt-labs/dbt-core/issues/7698))
- Improve warnings for constraints and materialization types ([#7335](https://github.com/dbt-labs/dbt-core/issues/7335))
- Incorrect paths used for "target" and "state" directories ([#7465](https://github.com/dbt-labs/dbt-core/issues/7465))
- fix StopIteration error when publication for project not found ([#7711](https://github.com/dbt-labs/dbt-core/issues/7711))
- Using version 0 works when resolving single model ([#7372](https://github.com/dbt-labs/dbt-core/issues/7372))
- Fix empty --warn-error-options error message ([#7730](https://github.com/dbt-labs/dbt-core/issues/7730))
- send sql header on contract enforcement ([#7714](https://github.com/dbt-labs/dbt-core/issues/7714))
- Fixed doc link in selector.py ([#7533](https://github.com/dbt-labs/dbt-core/issues/7533))
- Fix null-safe equals comparison via `equals` ([#7778](https://github.com/dbt-labs/dbt-core/issues/7778))
- Log PublicationArtifactAvailable even when partially parsing unchanged public models ([#7782](https://github.com/dbt-labs/dbt-core/issues/7782))
- fix RuntimeError when removing project dependency from dependencies.yml ([#7743](https://github.com/dbt-labs/dbt-core/issues/7743))
### Under the Hood
- Fix flaky test for --fail-fast ([#7744](https://github.com/dbt-labs/dbt-core/issues/7744))
- Create `add_from_artifact` to populate `state_relation` field of nodes ([#7551](https://github.com/dbt-labs/dbt-core/issues/7551))
### Contributors
- [@dave-connors-3](https://github.com/dave-connors-3) ([#7738](https://github.com/dbt-labs/dbt-core/issues/7738))
- [@quazi-irfan](https://github.com/quazi-irfan) ([#7533](https://github.com/dbt-labs/dbt-core/issues/7533))
- [@sdebruyn](https://github.com/sdebruyn) ([#7082](https://github.com/dbt-labs/dbt-core/issues/7082))
- [@stu-k](https://github.com/stu-k) ([#7299](https://github.com/dbt-labs/dbt-core/issues/7299), [#7551](https://github.com/dbt-labs/dbt-core/issues/7551))
## dbt-core 1.6.0-b2 - May 25, 2023
### Features
- Added warnings for model and ref deprecations ([#7433](https://github.com/dbt-labs/dbt-core/issues/7433))
- Update drop_relation macro to allow for configuration of drop statement separately from object name ([#7625](https://github.com/dbt-labs/dbt-core/issues/7625))
- accept publications in dbt.invoke ([#7372](https://github.com/dbt-labs/dbt-core/issues/7372))
### Fixes
- Honor `--skip-profile-setup` parameter when inside an existing project ([#7594](https://github.com/dbt-labs/dbt-core/issues/7594))
- Fix: Relative project paths weren't working with deps ([#7491](https://github.com/dbt-labs/dbt-core/issues/7491))
- Exclude password fields from Jinja rendering. ([#7629](https://github.com/dbt-labs/dbt-core/issues/7629))
- Add --target-path to more CLI subcommands ([#7646](https://github.com/dbt-labs/dbt-core/issues/7646))
- Stringify flag paths for Jinja context ([#7495](https://github.com/dbt-labs/dbt-core/issues/7495))
- write run_results.json for run operation ([#7502](https://github.com/dbt-labs/dbt-core/issues/7502))
### Under the Hood
- Add ability to instantiate Flags class from dict ([#7607](https://github.com/dbt-labs/dbt-core/issues/7607))
- Add other relation to reffable nodes ([#7550](https://github.com/dbt-labs/dbt-core/issues/7550))
- Move node patch method to schema parser patch_node_properties and refactor schema parsing ([#7430](https://github.com/dbt-labs/dbt-core/issues/7430))
- Remove legacy file logger code ([#NA](https://github.com/dbt-labs/dbt-core/issues/NA))
- Break up integration tests as a short term fix for Windows CI runs ([#7668](https://github.com/dbt-labs/dbt-core/issues/7668))
- Include null checks in utils test base ([#7670](https://github.com/dbt-labs/dbt-core/issues/7670))
- Write pub artifact to log ([#7372](https://github.com/dbt-labs/dbt-core/issues/7372))
### Dependencies
- Bump ubuntu from 23.04 to 23.10 ([#7675](https://github.com/dbt-labs/dbt-core/pull/7675))
- ([#7681](https://github.com/dbt-labs/dbt-core/pull/7681))
### Contributors
- [@dradnan89@hotmail.com](https://github.com/dradnan89@hotmail.com) ([#7681](https://github.com/dbt-labs/dbt-core/pull/7681))
- [@dwreeves](https://github.com/dwreeves) ([#7646](https://github.com/dbt-labs/dbt-core/issues/7646))
- [@iknox-fa](https://github.com/iknox-fa) ([#7491](https://github.com/dbt-labs/dbt-core/issues/7491), [#NA](https://github.com/dbt-labs/dbt-core/issues/NA))
- [@sdebruyn](https://github.com/sdebruyn) ([#7670](https://github.com/dbt-labs/dbt-core/issues/7670))
- [@stu-k](https://github.com/stu-k) ([#7607](https://github.com/dbt-labs/dbt-core/issues/7607), [#7550](https://github.com/dbt-labs/dbt-core/issues/7550))
## dbt-core 1.6.0-b1 - May 12, 2023
### Features
- Skip catalog generation ([#6980](https://github.com/dbt-labs/dbt-core/issues/6980))
- Publication artifacts and cross-project ref ([#7227](https://github.com/dbt-labs/dbt-core/issues/7227))
- Add graph structure summaries to target path output ([#7357](https://github.com/dbt-labs/dbt-core/issues/7357))
- Allow duplicate manifest node (models, seeds, analyses, snapshots) names across packages ([#7446](https://github.com/dbt-labs/dbt-core/issues/7446))
- Detect breaking changes to enforced constraints ([#7065](https://github.com/dbt-labs/dbt-core/issues/7065))
- Check for project dependency cycles ([#7468](https://github.com/dbt-labs/dbt-core/issues/7468))
### Fixes
- Persist timing info in run results for failed nodes ([#5476](https://github.com/dbt-labs/dbt-core/issues/5476))
- fix typo in unpacking statically parsed ref ([#7364](https://github.com/dbt-labs/dbt-core/issues/7364))
- safe version attribute access in _check_resource_uniqueness ([#7375](https://github.com/dbt-labs/dbt-core/issues/7375))
- Fix dbt command missing target-path param ([# 7411](https://github.com/dbt-labs/dbt-core/issues/ 7411))
- Fix v0 ref resolution ([#7408](https://github.com/dbt-labs/dbt-core/issues/7408))
- Add --target-path to dbt snapshot command. ([#7418](https://github.com/dbt-labs/dbt-core/issues/7418))
- dbt build selection of tests' descendants ([#7289](https://github.com/dbt-labs/dbt-core/issues/7289))
- fix groupable node partial parsing, raise DbtReferenceError at runtime for safety ([#7437](https://github.com/dbt-labs/dbt-core/issues/7437))
- Fix partial parsing of latest_version changes for downstream references ([#7369](https://github.com/dbt-labs/dbt-core/issues/7369))
- Use "add_node" to update depends_on.nodes ([#7453](https://github.com/dbt-labs/dbt-core/issues/7453))
- Fix var precedence in configs: root vars override package vars ([#6705](https://github.com/dbt-labs/dbt-core/issues/6705))
- Fix inverted `--print/--no-print` flag ([#7517](https://github.com/dbt-labs/dbt-core/issues/7517))
- Back-compat for previous return type of 'collect_freshness' macro ([#7489](https://github.com/dbt-labs/dbt-core/issues/7489))
- print model version in dbt show if specified ([#7407](https://github.com/dbt-labs/dbt-core/issues/7407))
- enable dbt show for seeds ([#7273](https://github.com/dbt-labs/dbt-core/issues/7273))
- push down limit filtering to adapter ([#7390](https://github.com/dbt-labs/dbt-core/issues/7390))
- Allow missing `profiles.yml` for `dbt deps` and `dbt init` ([#7511](https://github.com/dbt-labs/dbt-core/issues/7511))
- `run_results.json` is now written after every node completes. ([#7302](https://github.com/dbt-labs/dbt-core/issues/7302))
- Do not rewrite manifest.json during 'docs serve' command ([#7553](https://github.com/dbt-labs/dbt-core/issues/7553))
- Pin protobuf to greater than 4.0.0 ([#7565](https://github.com/dbt-labs/dbt-core/issues/7565))
- inject sql header in query for show ([#7413](https://github.com/dbt-labs/dbt-core/issues/7413))
- Pin urllib3 to ~=1.0 ([#7573](https://github.com/dbt-labs/dbt-core/issues/7573))
- Throw error for duplicated versioned and unversioned models ([#7487](https://github.com/dbt-labs/dbt-core/issues/7487))
### Under the Hood
- Update docs link in ContractBreakingChangeError message ([#7366](https://github.com/dbt-labs/dbt-core/issues/7366))
- Reduce memory footprint of cached statement results. ([#7281](https://github.com/dbt-labs/dbt-core/issues/7281))
- Remove noisy parsing events: GenericTestFileParse, MacroFileParse, Note events for static model parsing ([#6671](https://github.com/dbt-labs/dbt-core/issues/6671))
- Update --help text for cache-related parameters ([#7381](https://github.com/dbt-labs/dbt-core/issues/7381))
- Small UX improvements to model versions: Support defining latest_version in unsuffixed file by default. Notify on unpinned ref when a prerelease version is available. ([#7443](https://github.com/dbt-labs/dbt-core/issues/7443))
### Contributors
- [@AndyBys](https://github.com/AndyBys) ([#6980](https://github.com/dbt-labs/dbt-core/issues/6980))
- [@b-luu](https://github.com/b-luu) ([#7289](https://github.com/dbt-labs/dbt-core/issues/7289))
- [@dwreeves](https://github.com/dwreeves) ([#7418](https://github.com/dbt-labs/dbt-core/issues/7418))
- [@thomasgjerdekog](https://github.com/thomasgjerdekog) ([#7517](https://github.com/dbt-labs/dbt-core/issues/7517))
## dbt-core 1.6.0-a1 - April 17, 2023
## Previous Releases
For information on prior major and minor releases, see their changelogs:
* [1.6](https://github.com/dbt-labs/dbt-core/blob/1.6.latest/CHANGELOG.md)
* [1.5](https://github.com/dbt-labs/dbt-core/blob/1.5.latest/CHANGELOG.md)
* [1.4](https://github.com/dbt-labs/dbt-core/blob/1.4.latest/CHANGELOG.md)
* [1.3](https://github.com/dbt-labs/dbt-core/blob/1.3.latest/CHANGELOG.md)

View File

@@ -5,10 +5,10 @@
1. [About this document](#about-this-document)
2. [Getting the code](#getting-the-code)
3. [Setting up an environment](#setting-up-an-environment)
4. [Running `dbt` in development](#running-dbt-core-in-development)
4. [Running dbt-core in development](#running-dbt-core-in-development)
5. [Testing dbt-core](#testing)
6. [Debugging](#debugging)
7. [Adding a changelog entry](#adding-a-changelog-entry)
7. [Adding or modifying a changelog entry](#adding-or-modifying-a-changelog-entry)
8. [Submitting a Pull Request](#submitting-a-pull-request)
## About this document
@@ -113,7 +113,7 @@ When installed in this way, any changes you make to your local copy of the sourc
With your virtualenv activated, the `dbt` script should point back to the source code you've cloned on your machine. You can verify this by running `which dbt`. This command should show you a path to an executable in your virtualenv.
Configure your [profile](https://docs.getdbt.com/docs/configure-your-profile) as necessary to connect to your target databases. It may be a good idea to add a new profile pointing to a local Postgres instance, or a specific test sandbox within your data warehouse if appropriate.
Configure your [profile](https://docs.getdbt.com/docs/configure-your-profile) as necessary to connect to your target databases. It may be a good idea to add a new profile pointing to a local Postgres instance, or a specific test sandbox within your data warehouse if appropriate. Make sure to create a profile before running integration tests.
## Testing
@@ -171,12 +171,10 @@ Finally, you can also run a specific test or group of tests using [`pytest`](htt
```sh
# run all unit tests in a file
python3 -m pytest test/unit/test_graph.py
python3 -m pytest tests/unit/test_graph.py
# run a specific unit test
python3 -m pytest test/unit/test_graph.py::GraphTest::test__dependency_list
# run specific Postgres integration tests (old way)
python3 -m pytest -m profile_postgres test/integration/074_postgres_unlogged_table_tests
# run specific Postgres integration tests (new way)
python3 -m pytest tests/unit/test_graph.py::GraphTest::test__dependency_list
# run specific Postgres functional tests
python3 -m pytest tests/functional/sources
```
@@ -185,9 +183,8 @@ python3 -m pytest tests/functional/sources
### Unit, Integration, Functional?
Here are some general rules for adding tests:
* unit tests (`test/unit` & `tests/unit`) dont need to access a database; "pure Python" tests should be written as unit tests
* functional tests (`test/integration` & `tests/functional`) cover anything that interacts with a database, namely adapter
* *everything in* `test/*` *is being steadily migrated to* `tests/*`
* unit tests (`tests/unit`) dont need to access a database; "pure Python" tests should be written as unit tests
* functional tests (`tests/functional`) cover anything that interacts with a database, namely adapter
## Debugging

View File

@@ -3,13 +3,13 @@
# See `/docker` for a generic and production-ready docker file
##
FROM ubuntu:23.10
FROM ubuntu:22.04
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
software-properties-common \
software-properties-common gpg-agent \
&& add-apt-repository ppa:git-core/ppa -y \
&& apt-get dist-upgrade -y \
&& apt-get install -y --no-install-recommends \
@@ -30,8 +30,8 @@ RUN apt-get update \
unixodbc-dev \
&& add-apt-repository ppa:deadsnakes/ppa \
&& apt-get install -y \
python \
python-dev \
python-is-python3 \
python-dev-is-python3 \
python3-pip \
python3.8 \
python3.8-dev \

View File

@@ -289,6 +289,17 @@ class BaseAdapter(metaclass=AdapterMeta):
"""
return self.connections.execute(sql=sql, auto_begin=auto_begin, fetch=fetch, limit=limit)
def validate_sql(self, sql: str) -> AdapterResponse:
"""Submit the given SQL to the engine for validation, but not execution.
This should throw an appropriate exception if the input SQL is invalid, although
in practice that will generally be handled by delegating to an existing method
for execution and allowing the error handler to take care of the rest.
:param str sql: The sql to validate
"""
raise NotImplementedError("`validate_sql` is not implemented for this adapter!")
@available.parse(lambda *a, **k: [])
def get_column_schema_from_query(self, sql: str) -> List[BaseColumn]:
"""Get a list of the Columns with names and data types from the given sql."""
@@ -384,7 +395,7 @@ class BaseAdapter(metaclass=AdapterMeta):
return {
self.Relation.create_from(self.config, node).without_identifier()
for node in manifest.nodes.values()
if (node.is_relational and not node.is_ephemeral_model)
if (node.is_relational and not node.is_ephemeral_model and not node.is_external_node)
}
def _get_catalog_schemas(self, manifest: Manifest) -> SchemaSearchMap:
@@ -415,7 +426,7 @@ class BaseAdapter(metaclass=AdapterMeta):
return info_schema_name_map
def _relations_cache_for_schemas(
self, manifest: Manifest, cache_schemas: Set[BaseRelation] = None
self, manifest: Manifest, cache_schemas: Optional[Set[BaseRelation]] = None
) -> None:
"""Populate the relations cache for the given schemas. Returns an
iterable of the schemas populated, as strings.
@@ -451,7 +462,7 @@ class BaseAdapter(metaclass=AdapterMeta):
self,
manifest: Manifest,
clear: bool = False,
required_schemas: Set[BaseRelation] = None,
required_schemas: Optional[Set[BaseRelation]] = None,
) -> None:
"""Run a query that gets a populated cache of the relations in the
database and set the cache on this adapter.
@@ -785,7 +796,6 @@ class BaseAdapter(metaclass=AdapterMeta):
schema: str,
identifier: str,
) -> List[BaseRelation]:
matches = []
search = self._make_match_kwargs(database, schema, identifier)
@@ -986,7 +996,7 @@ class BaseAdapter(metaclass=AdapterMeta):
manifest: Optional[Manifest] = None,
project: Optional[str] = None,
context_override: Optional[Dict[str, Any]] = None,
kwargs: Dict[str, Any] = None,
kwargs: Optional[Dict[str, Any]] = None,
text_only_columns: Optional[Iterable[str]] = None,
) -> AttrDict:
"""Look macro_name up in the manifest and execute its results.
@@ -1063,7 +1073,6 @@ class BaseAdapter(metaclass=AdapterMeta):
schemas: Set[str],
manifest: Manifest,
) -> agate.Table:
kwargs = {"information_schema": information_schema, "schemas": schemas}
table = self.execute_macro(
GET_CATALOG_MACRO_NAME,
@@ -1346,7 +1355,8 @@ class BaseAdapter(metaclass=AdapterMeta):
rendered_column_constraints = []
for v in raw_columns.values():
rendered_column_constraint = [f"{v['name']} {v['data_type']}"]
col_name = cls.quote(v["name"]) if v.get("quote") else v["name"]
rendered_column_constraint = [f"{col_name} {v['data_type']}"]
for con in v.get("constraints", None):
constraint = cls._parse_column_constraint(con)
c = cls.process_parsed_constraint(constraint, cls.render_column_constraint)
@@ -1452,7 +1462,6 @@ join diff_count using (id)
def catch_as_completed(
futures, # typing: List[Future[agate.Table]]
) -> Tuple[agate.Table, List[Exception]]:
# catalogs: agate.Table = agate.Table(rows=[])
tables: List[agate.Table] = []
exceptions: List[Exception] = []

View File

@@ -9,10 +9,11 @@ from dbt.adapters.base.plugin import AdapterPlugin
from dbt.adapters.protocol import AdapterConfig, AdapterProtocol, RelationProtocol
from dbt.contracts.connection import AdapterRequiredConfig, Credentials
from dbt.events.functions import fire_event
from dbt.events.types import AdapterImportError, PluginLoadError
from dbt.events.types import AdapterImportError, PluginLoadError, AdapterRegistered
from dbt.exceptions import DbtInternalError, DbtRuntimeError
from dbt.include.global_project import PACKAGE_PATH as GLOBAL_PROJECT_PATH
from dbt.include.global_project import PROJECT_NAME as GLOBAL_PROJECT_NAME
from dbt.semver import VersionSpecifier
Adapter = AdapterProtocol
@@ -89,7 +90,13 @@ class AdapterContainer:
def register_adapter(self, config: AdapterRequiredConfig) -> None:
adapter_name = config.credentials.type
adapter_type = self.get_adapter_class_by_name(adapter_name)
adapter_version = import_module(f".{adapter_name}.__version__", "dbt.adapters").version
adapter_version_specifier = VersionSpecifier.from_version_string(
adapter_version
).to_version_string()
fire_event(
AdapterRegistered(adapter_name=adapter_name, adapter_version=adapter_version_specifier)
)
with self.lock:
if adapter_name in self.adapters:
# this shouldn't really happen...

Some files were not shown because too many files have changed in this diff Show More