Compare commits

...

9 Commits

Author SHA1 Message Date
Michelle Ark
1b7d9b5704 [Tidy first] move microbatch compilation to .compile method (#11063) 2024-11-27 19:08:36 -05:00
Quigley Malcolm
c3d87b89fb Add batch context object to microbatch jinja context (#11031)
* Add `batch_id` to jinja context of microbatch batches

* Add changie doc

* Update `format_batch_start` to assume `batch_start` is always provided

* Add "runtime only" property `batch_context` to `ModelNode`

By it being "runtime only" we mean that it doesn't exist on the artifact
and thus won't be written out to the manifest artifact.

* Begin populating `batch_context` during materialization execution for microbatch batches

* Fix circular import

* Fixup MicrobatchBuilder.batch_id property method

* Ensure MicrobatchModelRunner doesn't double compile batches

We were compiling the node for each batch _twice_. Besides making microbatch
models more expensive than they needed to be, double compiling wasn't
causing any issue. However the first compilation was happening _before_ we
had added the batch context information to the model node for the batch. This
was leading to models which try to access the `batch_context` information on the
model to blow up, which was undesirable. As such, we've now gone and skipped
the first compilation. We've done this similar to how SavedQuery nodes skip
compilation.

* Add `__post_serialize__` method to `BatchContext` to ensure correct dict shape

This is weird, but necessary, I apologize. Mashumaro handles the
dictification of this class via a compile time generated `to_dict`
method based off of the _typing_ of th class. By default `datetime`
types are converted to strings. We don't want that, we want them to
stay datetimes.

* Update tests to check for `batch_context`

* Update `resolve_event_time_filter` to use new `batch_context`

* Stop testing for batchless compiled code for microbatch models

In 45daec72f4 we stopped an extra compilation
that was happening per batch prior to the batch_context being loaded. Stopping
this extra compilation means that compiled sql for the microbatch model without
the event time filter / batch context is no longer produced. We have discussed
this and _believe_ it is okay given that this is a new node type that has not
hit GA yet.

* Rename `ModelNode.batch_context` to `ModelNode.batch`

* Rename `build_batch_context` to `build_jinja_context_for_batch`

The name `build_batch_context` was confusing as
1) We have a `BatchContext` object, which the method was not building
2) The method builds the jinja context for the batch
As such it felt appropriate to rename the method to more accurately
communicate what it does.

* Rename test macro `invalid_batch_context_macro_sql` to `invalid_batch_jinja_context_macro_sql`

This rename was to make it more clear that the jinja context for a
batch was being checked, as a batch_context has a slightly different
connotation.

* Update changie doc
2024-11-27 16:06:41 -06:00
Quigley Malcolm
0f084e16ca Rename internal batch_info variable to previous_batch_results (#11056)
* Rename `batch_info` to `previous_batch_results`

* Exclude `previous_batch_results` from serialization of model node to avoid jinja context bloat

* Drop `previous_batch_results` key from `test_manifest.py` unit tests

In 4050e377ec we began excluding
`previous_batch_results` from the serialized representation of the
ModelNode. As such, we no longer need to check for it in `test_manifest.py`.
2024-11-27 10:46:45 -06:00
Apoorv Mehrotra
3464be7f70 Fixes dbt retry does not respect --threads (#10591) 2024-11-26 11:21:46 -08:00
Gerda Shank
407f6caa1c Pin mashumaro to <3.15 (#11046) 2024-11-25 10:49:17 -05:00
Peter Webb
ad575ec699 Add New Config Properties and Schema for Snapshot Hard Deletes (#10972)
* Add changelog entry.

* Update schemas and test fixtures for new snapshot meta-column

* Add back comment.
2024-11-21 18:15:30 -05:00
Kshitij Aranke
f582ac2488 Fix #11012: Catch DbtRuntimeError for hooks (#11023) 2024-11-21 22:27:45 +00:00
Gerda Shank
f5f0735d00 Bump libpq-dev to 13.18-0+deb11u1 in docker/Dockerfile (#11029) 2024-11-21 17:24:53 -05:00
FishtownBuildBot
3abf575fa6 Cleanup main after cutting new 1.9.latest branch (#11027)
* Clean up changelog on main

* Bumping version to 1.10.0a1

* Code quality cleanup

* add 1.8,1.9 link

---------

Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com>
2024-11-21 15:54:56 -06:00
170 changed files with 276 additions and 1357 deletions

View File

@@ -1,5 +1,5 @@
[bumpversion]
current_version = 1.9.0b4
current_version = 1.10.0a1
parse = (?P<major>[\d]+) # major version number
\.(?P<minor>[\d]+) # minor version number
\.(?P<patch>[\d]+) # patch version number

View File

@@ -1,136 +0,0 @@
## dbt-core 1.9.0-b1 - October 01, 2024
### Breaking Changes
- Fix changing the current working directory when using dpt deps, clean and init. ([#8997](https://github.com/dbt-labs/dbt-core/issues/8997))
### Features
- serialize inferred primary key ([#9824](https://github.com/dbt-labs/dbt-core/issues/9824))
- Add unit_test: selection method ([#10053](https://github.com/dbt-labs/dbt-core/issues/10053))
- Maximally parallelize dbt clone in clone command" ([#7914](https://github.com/dbt-labs/dbt-core/issues/7914))
- Add --host flag to dbt docs serve, defaulting to '127.0.0.1' ([#10229](https://github.com/dbt-labs/dbt-core/issues/10229))
- Update data_test to accept arbitrary config options ([#10197](https://github.com/dbt-labs/dbt-core/issues/10197))
- add pre_model and post_model hook calls to data and unit tests to be able to provide extra config options ([#10198](https://github.com/dbt-labs/dbt-core/issues/10198))
- add --empty value to jinja context as flags.EMPTY ([#10317](https://github.com/dbt-labs/dbt-core/issues/10317))
- Warning message for snapshot timestamp data types ([#10234](https://github.com/dbt-labs/dbt-core/issues/10234))
- Support cumulative_type_params & sub-daily granularities in semantic manifest. ([#10360](https://github.com/dbt-labs/dbt-core/issues/10360))
- Add time_granularity to metric spec. ([#10376](https://github.com/dbt-labs/dbt-core/issues/10376))
- Support standard schema/database fields for snapshots ([#10301](https://github.com/dbt-labs/dbt-core/issues/10301))
- Support ref and source in foreign key constraint expressions, bump dbt-common minimum to 1.6 ([#8062](https://github.com/dbt-labs/dbt-core/issues/8062))
- Support new semantic layer time spine configs to enable sub-daily granularity. ([#10475](https://github.com/dbt-labs/dbt-core/issues/10475))
- Include models that depend on changed vars in state:modified, add state:modified.vars selection method ([#4304](https://github.com/dbt-labs/dbt-core/issues/4304))
- Add support for behavior flags ([#10618](https://github.com/dbt-labs/dbt-core/issues/10618))
- Enable `--resource-type` and `--exclude-resource-type` CLI flags and environment variables for `dbt test` ([#10656](https://github.com/dbt-labs/dbt-core/issues/10656))
- Allow configuring snapshot column names ([#10185](https://github.com/dbt-labs/dbt-core/issues/10185))
- Add custom_granularities to YAML spec for time spines. ([#9265](https://github.com/dbt-labs/dbt-core/issues/9265))
- Add basic functionality for creating microbatch incremental models ([#9490](https://github.com/dbt-labs/dbt-core/issues/9490), [#10635](https://github.com/dbt-labs/dbt-core/issues/10635), [#10637](https://github.com/dbt-labs/dbt-core/issues/10637), [#10638](https://github.com/dbt-labs/dbt-core/issues/10638), [#10636](https://github.com/dbt-labs/dbt-core/issues/10636), [#10662](https://github.com/dbt-labs/dbt-core/issues/10662), [#10639](https://github.com/dbt-labs/dbt-core/issues/10639))
- Execute microbatch models in batches ([#10700](https://github.com/dbt-labs/dbt-core/issues/10700))
- Create 'skip_nodes_if_on_run_start_fails' behavior change flag ([#7387](https://github.com/dbt-labs/dbt-core/issues/7387))
- Allow snapshots to be defined in YAML. ([#10246](https://github.com/dbt-labs/dbt-core/issues/10246))
- Write microbatch compiled/run targets to separate files, one per batch ([#10714](https://github.com/dbt-labs/dbt-core/issues/10714))
- Track incremental_strategy as part of model_run tracking event ([#10761](https://github.com/dbt-labs/dbt-core/issues/10761))
- Support required 'begin' config for microbatch models ([#10701](https://github.com/dbt-labs/dbt-core/issues/10701))
- Parse-time validation of microbatch configs: require event_time, batch_size, lookback and validate input event_time ([#10709](https://github.com/dbt-labs/dbt-core/issues/10709))
- Added the --inline-direct parameter to 'dbt show' ([#10770](https://github.com/dbt-labs/dbt-core/issues/10770))
- Enable `retry` support for microbatch models ([#10715](https://github.com/dbt-labs/dbt-core/issues/10715), [#10729](https://github.com/dbt-labs/dbt-core/issues/10729))
- Use unrendered database and schema source properties during state:modified, behind state_modified_compare_more_unrendered_values behavoiur flag ([#9573](https://github.com/dbt-labs/dbt-core/issues/9573))
- Ensure microbatch models respect `full_refresh` model config ([#10785](https://github.com/dbt-labs/dbt-core/issues/10785))
- Adds validations for custom_granularities to ensure unique naming. ([#9265](https://github.com/dbt-labs/dbt-core/issues/9265))
### Fixes
- Remove unused check_new method ([#7586](https://github.com/dbt-labs/dbt-core/issues/7586))
- Test case for `merge_exclude_columns` ([#8267](https://github.com/dbt-labs/dbt-core/issues/8267))
- Convert "Skipping model due to fail_fast" message to DEBUG level ([#8774](https://github.com/dbt-labs/dbt-core/issues/8774))
- Restore previous behavior for --favor-state: only favor defer_relation if not selected in current command" ([#10107](https://github.com/dbt-labs/dbt-core/issues/10107))
- Unit test fixture (csv) returns null for empty value ([#9881](https://github.com/dbt-labs/dbt-core/issues/9881))
- Fix json format log and --quiet for ls and jinja print by converting print call to fire events ([#8756](https://github.com/dbt-labs/dbt-core/issues/8756))
- Add resource type to saved_query ([#10168](https://github.com/dbt-labs/dbt-core/issues/10168))
- Fix: Order-insensitive unit test equality assertion for expected/actual with multiple nulls ([#10167](https://github.com/dbt-labs/dbt-core/issues/10167))
- Renaming or removing a contracted model should raise a BreakingChange warning/error ([#10116](https://github.com/dbt-labs/dbt-core/issues/10116))
- prefer disabled project nodes to external node ([#10224](https://github.com/dbt-labs/dbt-core/issues/10224))
- Fix issues with selectors and inline nodes ([#8943](https://github.com/dbt-labs/dbt-core/issues/8943), [#9269](https://github.com/dbt-labs/dbt-core/issues/9269))
- Fix snapshot config to work in yaml files ([#4000](https://github.com/dbt-labs/dbt-core/issues/4000))
- Improve handling of error when loading schema file list ([#10284](https://github.com/dbt-labs/dbt-core/issues/10284))
- Use model alias for the CTE identifier generated during ephemeral materialization ([#5273](https://github.com/dbt-labs/dbt-core/issues/5273))
- Implement state:modified for saved queries ([#10294](https://github.com/dbt-labs/dbt-core/issues/10294))
- Saved Query node fail during skip ([#10029](https://github.com/dbt-labs/dbt-core/issues/10029))
- DOn't warn on `unit_test` config paths that are properly used ([#10311](https://github.com/dbt-labs/dbt-core/issues/10311))
- Fix setting `silence` of `warn_error_options` via `dbt_project.yaml` flags ([#10160](https://github.com/dbt-labs/dbt-core/issues/10160))
- Attempt to provide test fixture tables with all values to set types correctly for comparisong with source tables ([#10365](https://github.com/dbt-labs/dbt-core/issues/10365))
- Limit data_tests deprecation to root_project ([#9835](https://github.com/dbt-labs/dbt-core/issues/9835))
- CLI flags should take precedence over env var flags ([#10304](https://github.com/dbt-labs/dbt-core/issues/10304))
- Fix typing for artifact schemas ([#10442](https://github.com/dbt-labs/dbt-core/issues/10442))
- Fix over deletion of generated_metrics in partial parsing ([#10450](https://github.com/dbt-labs/dbt-core/issues/10450))
- Fix error constructing warn_error_options ([#10452](https://github.com/dbt-labs/dbt-core/issues/10452))
- Do not update varchar column definitions if a contract exists ([#10362](https://github.com/dbt-labs/dbt-core/issues/10362))
- fix all_constraints access, disabled node parsing of non-uniquely named resources ([#10509](https://github.com/dbt-labs/dbt-core/issues/10509))
- respect --quiet and --warn-error-options for flag deprecations ([#10105](https://github.com/dbt-labs/dbt-core/issues/10105))
- Propagate measure label when using create_metrics ([#10536](https://github.com/dbt-labs/dbt-core/issues/10536))
- Fix state:modified check for exports ([#10138](https://github.com/dbt-labs/dbt-core/issues/10138))
- Filter out empty nodes after graph selection to support consistent selection of nodes that depend on upstream public models ([#8987](https://github.com/dbt-labs/dbt-core/issues/8987))
- Late render pre- and post-hooks configs in properties / schema YAML files ([#10603](https://github.com/dbt-labs/dbt-core/issues/10603))
- Allow the use of env_var function in certain macros in which it was previously unavailable. ([#10609](https://github.com/dbt-labs/dbt-core/issues/10609))
- Remove deprecation for tests: to data_tests: change ([#10564](https://github.com/dbt-labs/dbt-core/issues/10564))
- Fix `--resource-type test` for `dbt list` and `dbt build` ([#10730](https://github.com/dbt-labs/dbt-core/issues/10730))
- Fix unit tests for incremental model with alias ([#10754](https://github.com/dbt-labs/dbt-core/issues/10754))
- Allow singular tests to be documented in properties.yml ([#9005](https://github.com/dbt-labs/dbt-core/issues/9005))
- Ignore --empty in unit test ref/source rendering ([#10516](https://github.com/dbt-labs/dbt-core/issues/10516))
- Ignore rendered jinja in configs for state:modified, behind state_modified_compare_more_unrendered_values behaviour flag ([#9564](https://github.com/dbt-labs/dbt-core/issues/9564))
- Improve performance of infer primary key ([#10781](https://github.com/dbt-labs/dbt-core/issues/10781))
- Attempt to skip saved query processing when no semantic manifest changes ([#10563](https://github.com/dbt-labs/dbt-core/issues/10563))
- Ensure dbt retry of microbatch models doesn't lose prior successful state ([#10800](https://github.com/dbt-labs/dbt-core/issues/10800))
### Docs
- Enable display of unit tests ([dbt-docs/#501](https://github.com/dbt-labs/dbt-docs/issues/501))
- Unit tests not rendering ([dbt-docs/#506](https://github.com/dbt-labs/dbt-docs/issues/506))
- Add support for Saved Query node ([dbt-docs/#486](https://github.com/dbt-labs/dbt-docs/issues/486))
- Fix npm security vulnerabilities as of June 2024 ([dbt-docs/#513](https://github.com/dbt-labs/dbt-docs/issues/513))
### Under the Hood
- Clear error message for Private package in dbt-core ([#10083](https://github.com/dbt-labs/dbt-core/issues/10083))
- Enable use of context in serialization ([#10093](https://github.com/dbt-labs/dbt-core/issues/10093))
- Make RSS high water mark measurement more accurate on Linux ([#10177](https://github.com/dbt-labs/dbt-core/issues/10177))
- Enable record filtering by type. ([#10240](https://github.com/dbt-labs/dbt-core/issues/10240))
- Remove IntermediateSnapshotNode ([#10326](https://github.com/dbt-labs/dbt-core/issues/10326))
- Additional logging for skipped ephemeral models ([#10389](https://github.com/dbt-labs/dbt-core/issues/10389))
- bump black to 24.3.0 ([#10454](https://github.com/dbt-labs/dbt-core/issues/10454))
- generate protos with protoc version 5.26.1 ([#10457](https://github.com/dbt-labs/dbt-core/issues/10457))
- Move from minimal-snowplow-tracker fork back to snowplow-tracker ([#8409](https://github.com/dbt-labs/dbt-core/issues/8409))
- Add group info to RunResultError, RunResultFailure, RunResultWarning log lines ([#](https://github.com/dbt-labs/dbt-core/issues/))
- Improve speed of tree traversal when finding children, increasing build speed for some selectors ([#10434](https://github.com/dbt-labs/dbt-core/issues/10434))
- Add test for sources tables with quotes ([#10582](https://github.com/dbt-labs/dbt-core/issues/10582))
- Additional type hints for `core/dbt/version.py` ([#10612](https://github.com/dbt-labs/dbt-core/issues/10612))
- Fix typing issues in core/dbt/contracts/sql.py ([#10614](https://github.com/dbt-labs/dbt-core/issues/10614))
- Fix type errors in `dbt/core/task/clean.py` ([#10616](https://github.com/dbt-labs/dbt-core/issues/10616))
- Add Snowplow tracking for behavior flag deprecations ([#10552](https://github.com/dbt-labs/dbt-core/issues/10552))
- Add test utility patch_microbatch_end_time for adapters testing ([#10713](https://github.com/dbt-labs/dbt-core/issues/10713))
- Replace `TestSelector` with `ResourceTypeSelector` ([#10718](https://github.com/dbt-labs/dbt-core/issues/10718))
- Standardize returning `ResourceTypeSelector` instances in `dbt list` and `dbt build` ([#10739](https://github.com/dbt-labs/dbt-core/issues/10739))
- Add group metadata info to LogModelResult and LogTestResult ([#10775](https://github.com/dbt-labs/dbt-core/issues/10775))
### Dependencies
- Remove logbook dependency ([#8027](https://github.com/dbt-labs/dbt-core/issues/8027))
- Increase supported version range for dbt-semantic-interfaces. Needed to support custom calendar features. ([#9265](https://github.com/dbt-labs/dbt-core/issues/9265))
### Security
- Explicitly bind to localhost in docs serve ([#10209](https://github.com/dbt-labs/dbt-core/issues/10209))
### Contributors
- [@McKnight-42](https://github.com/McKnight-42) ([#10197](https://github.com/dbt-labs/dbt-core/issues/10197), [#10198](https://github.com/dbt-labs/dbt-core/issues/10198))
- [@TowardOliver](https://github.com/TowardOliver) ([#10656](https://github.com/dbt-labs/dbt-core/issues/10656))
- [@aliceliu](https://github.com/aliceliu) ([#10536](https://github.com/dbt-labs/dbt-core/issues/10536), [#10138](https://github.com/dbt-labs/dbt-core/issues/10138))
- [@courtneyholcomb](https://github.com/courtneyholcomb) ([#10360](https://github.com/dbt-labs/dbt-core/issues/10360), [#10376](https://github.com/dbt-labs/dbt-core/issues/10376), [#10475](https://github.com/dbt-labs/dbt-core/issues/10475), [#9265](https://github.com/dbt-labs/dbt-core/issues/9265), [#9265](https://github.com/dbt-labs/dbt-core/issues/9265), [#9265](https://github.com/dbt-labs/dbt-core/issues/9265))
- [@dave-connors-3](https://github.com/dave-connors-3) ([#9824](https://github.com/dbt-labs/dbt-core/issues/9824))
- [@jeancochrane](https://github.com/jeancochrane) ([#5273](https://github.com/dbt-labs/dbt-core/issues/5273))
- [@katsugeneration](https://github.com/katsugeneration) ([#10754](https://github.com/dbt-labs/dbt-core/issues/10754))
- [@kevinneville](https://github.com/kevinneville) ([#7586](https://github.com/dbt-labs/dbt-core/issues/7586))
- [@nakamichiworks](https://github.com/nakamichiworks) ([#10442](https://github.com/dbt-labs/dbt-core/issues/10442))
- [@rariyama](https://github.com/rariyama) ([#8997](https://github.com/dbt-labs/dbt-core/issues/8997))
- [@scottgigante,nevdelap](https://github.com/scottgigante,nevdelap) ([#8774](https://github.com/dbt-labs/dbt-core/issues/8774))
- [@ttusing](https://github.com/ttusing) ([#10434](https://github.com/dbt-labs/dbt-core/issues/10434))

View File

@@ -1 +0,0 @@
## dbt-core 1.9.0-b2 - October 07, 2024

View File

@@ -1,38 +0,0 @@
## dbt-core 1.9.0-b3 - October 30, 2024
### Features
- Add `order_by` and `limit` fields to saved queries. ([#10531](https://github.com/dbt-labs/dbt-core/issues/10531))
- Enable specification of dbt_valid_to for current records ([#10187](https://github.com/dbt-labs/dbt-core/issues/10187))
- Enable use of multi-column unique key in snapshots ([#9992](https://github.com/dbt-labs/dbt-core/issues/9992))
- Ensure `--event-time-start` is before `--event-time-end` ([#10786](https://github.com/dbt-labs/dbt-core/issues/10786))
- Ensure microbatch models use same `current_time` value ([#10819](https://github.com/dbt-labs/dbt-core/issues/10819))
- Emit warning when microbatch model has no input with `event_time` config ([#10926](https://github.com/dbt-labs/dbt-core/issues/10926))
### Fixes
- Pass test user config to adapter pre_hook by explicitly adding test builder config to node ([#10484](https://github.com/dbt-labs/dbt-core/issues/10484))
- Handle edge cases when a specified `--event-time-end` is equivalent to the batch size truncated batch start time ([#10824](https://github.com/dbt-labs/dbt-core/issues/10824))
- Begin tracking execution time of microbatch model batches ([#10825](https://github.com/dbt-labs/dbt-core/issues/10825))
- Allow instances of generic data tests to be documented ([#2578](https://github.com/dbt-labs/dbt-core/issues/2578))
- Fix warnings for models referring to a deprecated model ([#10833](https://github.com/dbt-labs/dbt-core/issues/10833))
- Change `lookback` default from `0` to `1` to ensure better data completeness ([#10867](https://github.com/dbt-labs/dbt-core/issues/10867))
- Make `--event-time-start` and `--event-time-end` mutually required ([#10874](https://github.com/dbt-labs/dbt-core/issues/10874))
- Exclude hook result from results in on-run-end context ([#7387](https://github.com/dbt-labs/dbt-core/issues/7387))
- Implement partial parsing for all-yaml snapshots ([#10903](https://github.com/dbt-labs/dbt-core/issues/10903))
- Restore source quoting behaviour when quoting config provided in dbt_project.yml ([#10892](https://github.com/dbt-labs/dbt-core/issues/10892))
- Fix bug when referencing deprecated models ([#10915](https://github.com/dbt-labs/dbt-core/issues/10915))
- Fix 'model' jinja context variable type to dict ([#10927](https://github.com/dbt-labs/dbt-core/issues/10927))
- Take `end_time` for batches to the ceiling to handle edge case where `event_time` column is a date ([#10868](https://github.com/dbt-labs/dbt-core/issues/10868))
### Under the Hood
- Remove support and testing for Python 3.8, which is now EOL. ([#10861](https://github.com/dbt-labs/dbt-core/issues/10861))
### Dependencies
- Bump minimnum allowed dbt-adapters version to 1.8.0 ([#N/A](https://github.com/dbt-labs/dbt-core/issues/N/A))
### Contributors
- [@danlsn](https://github.com/danlsn) ([#10915](https://github.com/dbt-labs/dbt-core/issues/10915))
- [@plypaul](https://github.com/plypaul) ([#10531](https://github.com/dbt-labs/dbt-core/issues/10531))

View File

@@ -1,19 +0,0 @@
## dbt-core 1.9.0-b4 - November 06, 2024
### Features
- Emit debug logging event whenever artifacts are written ([#10937](https://github.com/dbt-labs/dbt-core/issues/10937))
- Support --empty for snapshots ([#10372](https://github.com/dbt-labs/dbt-core/issues/10372))
### Fixes
- Ensure KeyboardInterrupt/SystemExit halts microbatch model execution ([#10862](https://github.com/dbt-labs/dbt-core/issues/10862))
- Handle exceptions in `get_execution_status` more broadly to better ensure `run_results.json` gets written ([#10934](https://github.com/dbt-labs/dbt-core/issues/10934))
### Under the Hood
- Behavior change for mf timespine without yaml configuration ([#10959](https://github.com/dbt-labs/dbt-core/issues/10959))
- Behavior change for cumulative metric type param ([#10960](https://github.com/dbt-labs/dbt-core/issues/10960))
### Contributors
- [@DevonFulcher](https://github.com/DevonFulcher) ([#10959](https://github.com/dbt-labs/dbt-core/issues/10959), [#10960](https://github.com/dbt-labs/dbt-core/issues/10960))

View File

@@ -1,6 +0,0 @@
kind: Breaking Changes
body: Fix changing the current working directory when using dpt deps, clean and init.
time: 2023-12-06T19:24:42.575372+09:00
custom:
Author: rariyama
Issue: "8997"

View File

@@ -1,6 +0,0 @@
kind: Dependencies
body: Remove logbook dependency
time: 2024-05-09T09:37:17.745129-05:00
custom:
Author: emmyoop
Issue: "8027"

View File

@@ -1,7 +0,0 @@
kind: Dependencies
body: Increase supported version range for dbt-semantic-interfaces. Needed to support
custom calendar features.
time: 2024-08-20T13:19:09.015225-07:00
custom:
Author: courtneyholcomb
Issue: "9265"

View File

@@ -1,6 +0,0 @@
kind: Dependencies
body: Bump minimnum allowed dbt-adapters version to 1.8.0
time: 2024-10-30T13:45:26.144328-05:00
custom:
Author: QMalcolm
Issue: N/A

View File

@@ -1,6 +0,0 @@
kind: Docs
body: Enable display of unit tests
time: 2024-03-11T14:03:44.490834-04:00
custom:
Author: gshank
Issue: "501"

View File

@@ -1,6 +0,0 @@
kind: Docs
body: Unit tests not rendering
time: 2024-05-01T02:10:50.987412+02:00
custom:
Author: aranke
Issue: "506"

View File

@@ -1,6 +0,0 @@
kind: Docs
body: Add support for Saved Query node
time: 2024-05-16T22:30:36.206492-07:00
custom:
Author: ChenyuLInx
Issue: "486"

View File

@@ -1,6 +0,0 @@
kind: Docs
body: Fix npm security vulnerabilities as of June 2024
time: 2024-06-13T15:10:48.301989+01:00
custom:
Author: aranke
Issue: "513"

View File

@@ -1,6 +0,0 @@
kind: Features
body: serialize inferred primary key
time: 2024-05-06T17:56:42.757673-05:00
custom:
Author: dave-connors-3
Issue: "9824"

View File

@@ -1,6 +0,0 @@
kind: Features
body: 'Add unit_test: selection method'
time: 2024-05-07T16:27:17.047585-04:00
custom:
Author: michelleark
Issue: "10053"

View File

@@ -1,7 +0,0 @@
kind: Features
body: Maximally parallelize dbt clone
in clone command"
time: 2024-05-22T00:03:09.765977-04:00
custom:
Author: michelleark
Issue: "7914"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Add --host flag to dbt docs serve, defaulting to '127.0.0.1'
time: 2024-05-27T12:44:05.040843-04:00
custom:
Author: michelleark
Issue: "10229"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Update data_test to accept arbitrary config options
time: 2024-05-31T15:08:16.431966-05:00
custom:
Author: McKnight-42
Issue: "10197"

View File

@@ -1,6 +0,0 @@
kind: Features
body: add pre_model and post_model hook calls to data and unit tests to be able to provide extra config options
time: 2024-06-06T11:23:34.758675-05:00
custom:
Author: McKnight-42
Issue: "10198"

View File

@@ -1,6 +0,0 @@
kind: Features
body: add --empty value to jinja context as flags.EMPTY
time: 2024-06-17T10:39:48.275801-04:00
custom:
Author: michelleark
Issue: "10317"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Warning message for snapshot timestamp data types
time: 2024-06-21T14:16:35.717637-04:00
custom:
Author: gshank
Issue: "10234"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Support cumulative_type_params & sub-daily granularities in semantic manifest.
time: 2024-06-25T09:51:07.983248-07:00
custom:
Author: courtneyholcomb
Issue: "10360"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Add time_granularity to metric spec.
time: 2024-06-27T16:29:53.500917-07:00
custom:
Author: courtneyholcomb
Issue: "10376"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Support standard schema/database fields for snapshots
time: 2024-07-12T21:45:46.06011-04:00
custom:
Author: gshank
Issue: "10301"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Support ref and source in foreign key constraint expressions, bump dbt-common minimum to 1.6
time: 2024-07-19T16:18:41.434278-04:00
custom:
Author: michelleark
Issue: "8062"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Support new semantic layer time spine configs to enable sub-daily granularity.
time: 2024-07-22T20:22:38.258249-07:00
custom:
Author: courtneyholcomb
Issue: "10475"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Add `order_by` and `limit` fields to saved queries.
time: 2024-08-06T14:48:59.035914-07:00
custom:
Author: plypaul
Issue: "10531"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Add support for behavior flags
time: 2024-08-29T13:53:20.16122-04:00
custom:
Author: mikealfare
Issue: "10618"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Enable `--resource-type` and `--exclude-resource-type` CLI flags and environment variables for `dbt test`
time: 2024-09-03T13:24:28.592837+01:00
custom:
Author: TowardOliver dbeatty10
Issue: "10656"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Allow configuring snapshot column names
time: 2024-09-03T15:41:33.167097-04:00
custom:
Author: gshank
Issue: "10185"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Add custom_granularities to YAML spec for time spines.
time: 2024-09-04T18:23:20.234952-07:00
custom:
Author: courtneyholcomb
Issue: "9265"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Add basic functionality for creating microbatch incremental models
time: 2024-09-11T12:10:29.822189-05:00
custom:
Author: MichelleArk QMalcolm
Issue: 9490 10635 10637 10638 10636 10662 10639

View File

@@ -1,6 +0,0 @@
kind: Features
body: Execute microbatch models in batches
time: 2024-09-13T23:21:11.935434-04:00
custom:
Author: michelleark
Issue: "10700"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Create 'skip_nodes_if_on_run_start_fails' behavior change flag
time: 2024-09-18T16:29:59.268422+01:00
custom:
Author: aranke
Issue: "7387"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Allow snapshots to be defined in YAML.
time: 2024-09-20T11:04:47.703117-04:00
custom:
Author: peterallenwebb
Issue: "10246"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Write microbatch compiled/run targets to separate files, one per batch
time: 2024-09-20T17:24:19.219556+01:00
custom:
Author: michelleark
Issue: "10714"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Track incremental_strategy as part of model_run tracking event
time: 2024-09-23T15:53:35.898928+01:00
custom:
Author: michelleark
Issue: "10761"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Support required 'begin' config for microbatch models
time: 2024-09-23T15:59:03.924079+01:00
custom:
Author: michelleark
Issue: "10701"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Added the --inline-direct parameter to 'dbt show'
time: 2024-09-24T15:29:22.874496-04:00
custom:
Author: aranke peterallenwebb
Issue: "10770"

View File

@@ -1,7 +0,0 @@
kind: Features
body: 'Parse-time validation of microbatch configs: require event_time, batch_size,
lookback and validate input event_time'
time: 2024-09-24T15:46:39.83112+01:00
custom:
Author: michelleark
Issue: "10709"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Enable specification of dbt_valid_to for current records
time: 2024-09-25T12:08:55.926848-04:00
custom:
Author: gshank
Issue: "10187"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Enable `retry` support for microbatch models
time: 2024-09-25T16:50:02.105069-05:00
custom:
Author: QMalcolm MichelleArk
Issue: 10715 10729

View File

@@ -1,7 +0,0 @@
kind: Features
body: Use unrendered database and schema source properties during state:modified,
behind state_modified_compare_more_unrendered_values behavoiur flag
time: 2024-09-26T14:02:06.732068+01:00
custom:
Author: michelleark
Issue: "9573"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Adds validations for custom_granularities to ensure unique naming.
time: 2024-09-26T15:10:57.907694-07:00
custom:
Author: courtneyholcomb
Issue: "9265"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Ensure microbatch models respect `full_refresh` model config
time: 2024-09-26T15:32:10.202789-05:00
custom:
Author: QMalcolm
Issue: "10785"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Enable use of multi-column unique key in snapshots
time: 2024-10-01T13:40:51.297529-04:00
custom:
Author: gshank
Issue: "9992"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Ensure `--event-time-start` is before `--event-time-end`
time: 2024-10-03T17:05:29.984398-05:00
custom:
Author: QMalcolm
Issue: "10786"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Ensure microbatch models use same `current_time` value
time: 2024-10-07T11:58:53.460941-05:00
custom:
Author: QMalcolm
Issue: "10819"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Emit warning when microbatch model has no input with `event_time` config
time: 2024-10-28T17:34:19.195209-04:00
custom:
Author: michelleark
Issue: "10926"

View File

@@ -1,6 +0,0 @@
kind: Features
body: Emit debug logging event whenever artifacts are written
time: 2024-10-29T18:17:28.321188-05:00
custom:
Author: QMalcolm
Issue: 10937

View File

@@ -1,6 +0,0 @@
kind: Features
body: Support --empty for snapshots
time: 2024-10-31T16:31:49.926164-04:00
custom:
Author: gshank
Issue: "10372"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Remove unused check_new method
time: 2023-06-01T20:41:57.556342+02:00
custom:
Author: kevinneville
Issue: "7586"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Test case for `merge_exclude_columns`
time: 2023-08-01T09:46:26.829362-06:00
custom:
Author: dbeatty10
Issue: "8267"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Convert "Skipping model due to fail_fast" message to DEBUG level
time: 2024-01-13T07:36:15.836294-00:00
custom:
Author: scottgigante,nevdelap
Issue: "8774"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: 'Restore previous behavior for --favor-state: only favor defer_relation if not
selected in current command"'
time: 2024-05-08T15:11:27.510912+02:00
custom:
Author: jtcohen6
Issue: "10107"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Unit test fixture (csv) returns null for empty value
time: 2024-05-09T09:14:11.772709-04:00
custom:
Author: michelleark
Issue: "9881"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Fix json format log and --quiet for ls and jinja print by converting print call
to fire events
time: 2024-05-16T15:39:13.896723-07:00
custom:
Author: ChenyuLInx
Issue: "8756"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Add resource type to saved_query
time: 2024-05-16T22:35:10.287514-07:00
custom:
Author: ChenyuLInx
Issue: "10168"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: 'Fix: Order-insensitive unit test equality assertion for expected/actual with
multiple nulls'
time: 2024-05-22T18:28:55.91733-04:00
custom:
Author: michelleark
Issue: "10167"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Renaming or removing a contracted model should raise a BreakingChange warning/error
time: 2024-05-23T20:42:51.033946-04:00
custom:
Author: michelleark
Issue: "10116"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: prefer disabled project nodes to external node
time: 2024-05-24T13:11:35.440443-04:00
custom:
Author: michelleark
Issue: "10224"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Fix issues with selectors and inline nodes
time: 2024-06-05T11:16:52.187667-04:00
custom:
Author: gshank
Issue: 8943 9269

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Fix snapshot config to work in yaml files
time: 2024-06-07T13:46:48.383215-04:00
custom:
Author: gshank
Issue: "4000"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Improve handling of error when loading schema file list
time: 2024-06-10T13:21:30.963371-04:00
custom:
Author: gshank
Issue: "10284"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Use model alias for the CTE identifier generated during ephemeral materialization
time: 2024-06-10T20:05:22.510814008Z
custom:
Author: jeancochrane
Issue: "5273"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Saved Query node fail during skip
time: 2024-06-12T12:42:56.329073-07:00
custom:
Author: ChenyuLInx
Issue: "10029"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Implement state:modified for saved queries
time: 2024-06-12T15:21:39.851426-04:00
custom:
Author: gshank
Issue: "10294"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: DOn't warn on `unit_test` config paths that are properly used
time: 2024-06-13T18:31:17.486497-07:00
custom:
Author: QMalcolm
Issue: "10311"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Fix setting `silence` of `warn_error_options` via `dbt_project.yaml` flags
time: 2024-06-24T17:17:29.464865-07:00
custom:
Author: QMalcolm
Issue: "10160"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Attempt to provide test fixture tables with all values to set types correctly
for comparisong with source tables
time: 2024-06-25T17:17:37.514619-07:00
custom:
Author: versusfacit
Issue: "10365"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Limit data_tests deprecation to root_project
time: 2024-06-27T15:44:48.579869-04:00
custom:
Author: gshank
Issue: "9835"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: CLI flags should take precedence over env var flags
time: 2024-07-09T17:24:40.918977-04:00
custom:
Author: gshank
Issue: "10304"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Fix typing for artifact schemas
time: 2024-07-14T10:02:54.452099+09:00
custom:
Author: nakamichiworks
Issue: "10442"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Fix over deletion of generated_metrics in partial parsing
time: 2024-07-16T13:37:03.49651-04:00
custom:
Author: gshank
Issue: "10450"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Fix error constructing warn_error_options
time: 2024-07-16T17:14:27.837171-04:00
custom:
Author: gshank
Issue: "10452"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Do not update varchar column definitions if a contract exists
time: 2024-07-28T22:14:21.67712-04:00
custom:
Author: gshank
Issue: "10362"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: fix all_constraints access, disabled node parsing of non-uniquely named resources
time: 2024-07-31T09:51:52.751135-04:00
custom:
Author: michelleark gshank
Issue: "10509"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Propagate measure label when using create_metrics
time: 2024-08-06T17:21:10.265494-07:00
custom:
Author: aliceliu
Issue: "10536"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: respect --quiet and --warn-error-options for flag deprecations
time: 2024-08-06T19:48:43.399453-04:00
custom:
Author: michelleark
Issue: "10105"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Fix state:modified check for exports
time: 2024-08-13T15:42:35.471685-07:00
custom:
Author: aliceliu
Issue: "10138"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Filter out empty nodes after graph selection to support consistent selection of nodes that depend on upstream public models
time: 2024-08-16T14:08:07.426235-07:00
custom:
Author: jtcohen6
Issue: "8987"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Late render pre- and post-hooks configs in properties / schema YAML files
time: 2024-08-24T21:09:03.252733-06:00
custom:
Author: dbeatty10
Issue: "10603"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Allow the use of env_var function in certain macros in which it was previously
unavailable.
time: 2024-08-29T10:57:01.160613-04:00
custom:
Author: peterallenwebb
Issue: "10609"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: 'Remove deprecation for tests: to data_tests: change'
time: 2024-09-05T18:02:48.086421-04:00
custom:
Author: gshank
Issue: "10564"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Fix `--resource-type test` for `dbt list` and `dbt build`
time: 2024-09-17T17:44:46.121032-06:00
custom:
Author: dbeatty10
Issue: "10730"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Fix unit tests for incremental model with alias
time: 2024-09-22T13:35:27.991398741Z
custom:
Author: katsugeneration
Issue: "10754"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Allow singular tests to be documented in properties.yml
time: 2024-09-23T19:07:58.151069+01:00
custom:
Author: aranke
Issue: "9005"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Ignore --empty in unit test ref/source rendering
time: 2024-09-23T20:20:24.151285+01:00
custom:
Author: michelleark
Issue: "10516"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Ignore rendered jinja in configs for state:modified, behind state_modified_compare_more_unrendered_values
behaviour flag
time: 2024-09-25T13:10:28.490042+01:00
custom:
Author: michelleark
Issue: "9564"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Pass test user config to adapter pre_hook by explicitly adding test builder config to node
time: 2024-09-25T15:45:14.459598-07:00
custom:
Author: colin-rogers-dbt
Issue: "10484"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Improve performance of infer primary key
time: 2024-09-25T16:05:43.59536-04:00
custom:
Author: gshank
Issue: "10781"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Attempt to skip saved query processing when no semantic manifest changes
time: 2024-09-26T10:12:20.193453-04:00
custom:
Author: gshank
Issue: "10563"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Ensure dbt retry of microbatch models doesn't lose prior successful state
time: 2024-09-30T15:31:58.541656-05:00
custom:
Author: QMalcolm
Issue: "10800"

View File

@@ -1,7 +0,0 @@
kind: Fixes
body: Handle edge cases when a specified `--event-time-end` is equivalent to the batch
size truncated batch start time
time: 2024-10-04T13:36:30.357936-05:00
custom:
Author: QMalcolm
Issue: "10824"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Begin tracking execution time of microbatch model batches
time: 2024-10-04T16:39:08.464064-05:00
custom:
Author: QMalcolm
Issue: "10825"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Allow instances of generic data tests to be documented
time: 2024-10-14T21:21:35.767115+01:00
custom:
Author: aranke
Issue: "2578"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Fix warnings for models referring to a deprecated model
time: 2024-10-15T12:18:25.14525-04:00
custom:
Author: gshank
Issue: "10833"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Change `lookback` default from `0` to `1` to ensure better data completeness
time: 2024-10-17T13:48:57.805205-07:00
custom:
Author: QMalcolm
Issue: "10867"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Make `--event-time-start` and `--event-time-end` mutually required
time: 2024-10-17T14:53:57.149238-07:00
custom:
Author: QMalcolm
Issue: "10874"

View File

@@ -1,6 +0,0 @@
kind: Fixes
body: Ensure KeyboardInterrupt/SystemExit halts microbatch model execution
time: 2024-10-17T15:30:22.781854-07:00
custom:
Author: QMalcolm
Issue: "10862"

Some files were not shown because too many files have changed in this diff Show More