add python-dev-tools to dev-requirements

.bumpversion.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 1.7.18
+current_version = 1.7.0a1
 parse = (?P<major>[\d]+) # major version number
 	\.(?P<minor>[\d]+) # minor version number
 	\.(?P<patch>[\d]+) # patch version number

.changes/1.7.0.md

@@ -1,157 +0,0 @@
-## dbt-core 1.7.0 - November 02, 2023
-
-### Breaking Changes
-
-- Removed the FirstRunResultError and AfterFirstRunResultError event types, using the existing RunResultError in their place. ([#7963](https://github.com/dbt-labs/dbt-core/issues/7963))
-
-### Features
-
-- add log file of installed packages via dbt deps ([#6643](https://github.com/dbt-labs/dbt-core/issues/6643))
-- Enable re-population of metadata vars post-environment change during programmatic invocation ([#8010](https://github.com/dbt-labs/dbt-core/issues/8010))
-- Added support to configure a delimiter for a seed file, defaults to comma ([#3990](https://github.com/dbt-labs/dbt-core/issues/3990))
-- Allow specification of `create_metric: true` on measures ([#8125](https://github.com/dbt-labs/dbt-core/issues/8125))
-- Add node attributes related to compilation to run_results.json ([#7519](https://github.com/dbt-labs/dbt-core/issues/7519))
-- Add --no-inject-ephemeral-ctes flag for `compile` command, for usage by linting. ([#8480](https://github.com/dbt-labs/dbt-core/issues/8480))
-- Support configuration of semantic models with the addition of enable/disable and group enablement. ([#7968](https://github.com/dbt-labs/dbt-core/issues/7968))
-- Accept a `dbt-cloud` config in dbt_project.yml ([#8438](https://github.com/dbt-labs/dbt-core/issues/8438))
-- Support atomic replace in the global replace macro ([#8539](https://github.com/dbt-labs/dbt-core/issues/8539))
-- Use translate_type on data_type in model.columns in templates by default, remove no op `TYPE_LABELS` ([#8007](https://github.com/dbt-labs/dbt-core/issues/8007))
-- Add an option to generate static documentation ([#8614](https://github.com/dbt-labs/dbt-core/issues/8614))
-- Allow setting "access" as a config in addition to as a property ([#8383](https://github.com/dbt-labs/dbt-core/issues/8383))
-- Loosen typing requirement on renameable/replaceable relations to Iterable to allow adapters more flexibility in registering relation types, include docstrings as suggestions ([#8647](https://github.com/dbt-labs/dbt-core/issues/8647))
-- Add support for optional label in semantic_models, measures, dimensions and entities. ([#8595](https://github.com/dbt-labs/dbt-core/issues/8595), [#8755](https://github.com/dbt-labs/dbt-core/issues/8755))
-- Allow adapters to include package logs in dbt standard logging  ([#7859](https://github.com/dbt-labs/dbt-core/issues/7859))
-- Support storing test failures as views ([#6914](https://github.com/dbt-labs/dbt-core/issues/6914))
-- resolve packages with same git repo and unique subdirectory ([#5374](https://github.com/dbt-labs/dbt-core/issues/5374))
-- Add new ResourceReport event to record memory/cpu/io metrics ([#8342](https://github.com/dbt-labs/dbt-core/issues/8342))
-- Adding `date_spine` macro (and supporting macros) from dbt-utils to dbt-core ([#8172](https://github.com/dbt-labs/dbt-core/issues/8172))
-- Support `fill_nulls_with` and `join_to_timespine` for metric nodes ([#8593](https://github.com/dbt-labs/dbt-core/issues/8593), [#8755](https://github.com/dbt-labs/dbt-core/issues/8755))
-- Raise a warning when a contracted model has a numeric field without scale defined ([#8183](https://github.com/dbt-labs/dbt-core/issues/8183))
-- Added support for retrieving partial catalog information from a schema ([#8521](https://github.com/dbt-labs/dbt-core/issues/8521))
-- Add meta attribute to SemanticModels config ([#8511](https://github.com/dbt-labs/dbt-core/issues/8511))
-- Selectors with docs generate limits catalog generation ([#6014](https://github.com/dbt-labs/dbt-core/issues/6014))
-- Allow freshness to be determined via DBMS metadata for supported adapters ([#8704](https://github.com/dbt-labs/dbt-core/issues/8704))
-- Add support semantic layer SavedQuery node type ([#8594](https://github.com/dbt-labs/dbt-core/issues/8594))
-- Add exports to SavedQuery spec ([#8892](https://github.com/dbt-labs/dbt-core/issues/8892))
-
-### Fixes
-
-- Copy dir during `dbt deps` if symlink fails ([#7428](https://github.com/dbt-labs/dbt-core/issues/7428), [#8223](https://github.com/dbt-labs/dbt-core/issues/8223))
-- If --profile specified with dbt-init, create the project with the specified profile ([#6154](https://github.com/dbt-labs/dbt-core/issues/6154))
-- Fixed double-underline ([#5301](https://github.com/dbt-labs/dbt-core/issues/5301))
-- Copy target_schema from config into snapshot node ([#6745](https://github.com/dbt-labs/dbt-core/issues/6745))
-- Enable converting deprecation warnings to errors ([#8130](https://github.com/dbt-labs/dbt-core/issues/8130))
-- Add status to Parse Inline Error ([#8173](https://github.com/dbt-labs/dbt-core/issues/8173))
-- Ensure `warn_error_options` get serialized in `invocation_args_dict` ([#7694](https://github.com/dbt-labs/dbt-core/issues/7694))
-- Stop detecting materialization macros based on macro name ([#6231](https://github.com/dbt-labs/dbt-core/issues/6231))
-- Update `dbt deps` download retry logic to handle `EOFError` exceptions ([#6653](https://github.com/dbt-labs/dbt-core/issues/6653))
-- Improve handling of CTE injection with ephemeral models ([#8213](https://github.com/dbt-labs/dbt-core/issues/8213))
-- Fix unbound local variable error in `checked_agg_time_dimension_for_measure` ([#8230](https://github.com/dbt-labs/dbt-core/issues/8230))
-- Ensure runtime errors are raised for graph runnable tasks (compile, show, run, etc) ([#8166](https://github.com/dbt-labs/dbt-core/issues/8166))
-- Fix retry not working with log-file-max-bytes ([#8297](https://github.com/dbt-labs/dbt-core/issues/8297))
-- Add explicit support for integers for the show command ([#8153](https://github.com/dbt-labs/dbt-core/issues/8153))
-- Detect changes to model access, version, or latest_version in state:modified ([#8189](https://github.com/dbt-labs/dbt-core/issues/8189))
-- Add connection status into list of statuses for dbt debug ([#8350](https://github.com/dbt-labs/dbt-core/issues/8350))
-- fix fqn-selection for external versioned models ([#8374](https://github.com/dbt-labs/dbt-core/issues/8374))
-- Fix: DbtInternalError after model that previously ref'd external model is deleted ([#8375](https://github.com/dbt-labs/dbt-core/issues/8375))
-- Fix using list command with path selector and project-dir ([#8385](https://github.com/dbt-labs/dbt-core/issues/8385))
-- Remedy performance regression by only writing run_results.json once. ([#8360](https://github.com/dbt-labs/dbt-core/issues/8360))
-- Add support for swapping materialized views with tables/views and vice versa ([#8449](https://github.com/dbt-labs/dbt-core/issues/8449))
-- Turn breaking changes to contracted models into warnings for unversioned models ([#8384](https://github.com/dbt-labs/dbt-core/issues/8384), [#8282](https://github.com/dbt-labs/dbt-core/issues/8282))
-- Ensure parsing does not break when `window_groupings` is not specified for `non_additive_dimension` ([#8453](https://github.com/dbt-labs/dbt-core/issues/8453))
-- fix ambiguous reference error for tests and versions when model name is duplicated across packages ([#8327](https://github.com/dbt-labs/dbt-core/issues/8327), [#8493](https://github.com/dbt-labs/dbt-core/issues/8493))
-- Fix "Internal Error: Expected node <unique-id> not found in manifest" when depends_on set on ModelNodeArgs ([#8506](https://github.com/dbt-labs/dbt-core/issues/8506))
-- Fix snapshot success message ([#7583](https://github.com/dbt-labs/dbt-core/issues/7583))
-- Parse the correct schema version from manifest ([#8544](https://github.com/dbt-labs/dbt-core/issues/8544))
-- make version comparison insensitive to order ([#8571](https://github.com/dbt-labs/dbt-core/issues/8571))
-- Update metric helper functions to work with new semantic layer metrics ([#8134](https://github.com/dbt-labs/dbt-core/issues/8134))
-- Disallow cleaning paths outside current working directory ([#8318](https://github.com/dbt-labs/dbt-core/issues/8318))
-- Warn when --state == --target ([#8160](https://github.com/dbt-labs/dbt-core/issues/8160))
-- update dbt show to include limit in DWH query  ([#8496,](https://github.com/dbt-labs/dbt-core/issues/8496,), [#8417](https://github.com/dbt-labs/dbt-core/issues/8417))
-- Support quoted parameter list for MultiOption CLI options. ([#8598](https://github.com/dbt-labs/dbt-core/issues/8598))
-- Support global flags passed in after subcommands ([#6497](https://github.com/dbt-labs/dbt-core/issues/6497))
-- Lower bound of `8.0.2` for `click` ([#8683](https://github.com/dbt-labs/dbt-core/issues/8683))
-- Fixes test type edges filter ([#8692](https://github.com/dbt-labs/dbt-core/issues/8692))
-- semantic models in graph selection ([#8589](https://github.com/dbt-labs/dbt-core/issues/8589))
-- Support doc blocks in nested semantic model YAML ([#8509](https://github.com/dbt-labs/dbt-core/issues/8509))
-- avoid double-rendering sql_header in dbt show ([#8739](https://github.com/dbt-labs/dbt-core/issues/8739))
-- Fix tag selection for projects with semantic models ([#8749](https://github.com/dbt-labs/dbt-core/issues/8749))
-- Foreign key constraint on incremental model results in Database Error ([#8022](https://github.com/dbt-labs/dbt-core/issues/8022))
-- Support docs blocks on versioned model column descriptions ([#8540](https://github.com/dbt-labs/dbt-core/issues/8540))
-- Enable seeds to be handled from stored manifest data ([#6875](https://github.com/dbt-labs/dbt-core/issues/6875))
-- Override path-like args in dbt retry ([#8682](https://github.com/dbt-labs/dbt-core/issues/8682))
-- Group updates on unmodified nodes are handled gracefully for state:modified ([#8371](https://github.com/dbt-labs/dbt-core/issues/8371))
-- Partial parsing fix for adding groups and updating models at the same time ([#8697](https://github.com/dbt-labs/dbt-core/issues/8697))
-- Fix partial parsing not working for semantic model change ([#8859](https://github.com/dbt-labs/dbt-core/issues/8859))
-- Rework get_catalog implementation to retain previous adapter interface semantics ([#8846](https://github.com/dbt-labs/dbt-core/issues/8846))
-- Add back contract enforcement for temporary tables on postgres ([#8857](https://github.com/dbt-labs/dbt-core/issues/8857))
-- Add version to fqn when version==0 ([#8836](https://github.com/dbt-labs/dbt-core/issues/8836))
-- Fix cased comparison in catalog-retrieval function. ([#8939](https://github.com/dbt-labs/dbt-core/issues/8939))
-- Catalog queries now assign the correct type to materialized views ([#8864](https://github.com/dbt-labs/dbt-core/issues/8864))
-- Make relation filtering None-tolerant for maximal flexibility across adapters. ([#8974](https://github.com/dbt-labs/dbt-core/issues/8974))
-
-### Docs
-
-- Corrected spelling of "Partiton" ([dbt-docs/#8100](https://github.com/dbt-labs/dbt-docs/issues/8100))
-- Remove static SQL codeblock for metrics ([dbt-docs/#436](https://github.com/dbt-labs/dbt-docs/issues/436))
-- fixed comment util.py ([dbt-docs/#None](https://github.com/dbt-labs/dbt-docs/issues/None))
-- Fix newline escapes and improve formatting in docker README ([dbt-docs/#8211](https://github.com/dbt-labs/dbt-docs/issues/8211))
-- Display contract and column constraints on the model page ([dbt-docs/#433](https://github.com/dbt-labs/dbt-docs/issues/433))
-- Display semantic model details in docs ([dbt-docs/#431](https://github.com/dbt-labs/dbt-docs/issues/431))
-
-### Under the Hood
-
-- Switch from hologram to mashumaro jsonschema ([#8426](https://github.com/dbt-labs/dbt-core/issues/8426))
-- Refactor flaky test pp_versioned_models ([#7781](https://github.com/dbt-labs/dbt-core/issues/7781))
-- format exception from dbtPlugin.initialize ([#8152](https://github.com/dbt-labs/dbt-core/issues/8152))
-- A way to control maxBytes for a single dbt.log file ([#8199](https://github.com/dbt-labs/dbt-core/issues/8199))
-- Ref expressions with version can now be processed by the latest version of the high-performance dbt-extractor library. ([#7688](https://github.com/dbt-labs/dbt-core/issues/7688))
-- Bump manifest schema version to v11, freeze manifest v10 ([#8333](https://github.com/dbt-labs/dbt-core/issues/8333))
-- add tracking for plugin.get_nodes calls ([#8344](https://github.com/dbt-labs/dbt-core/issues/8344))
-- add internal flag: --no-partial-parse-file-diff to inform whether to compute a file diff during partial parsing ([#8363](https://github.com/dbt-labs/dbt-core/issues/8363))
-- Add return values to a number of functions for mypy ([#8389](https://github.com/dbt-labs/dbt-core/issues/8389))
-- Fix mypy warnings for ManifestLoader.load() ([#8401](https://github.com/dbt-labs/dbt-core/issues/8401))
-- Use python version 3.10.7 in Docker image. ([#8444](https://github.com/dbt-labs/dbt-core/issues/8444))
-- Re-organize jinja macros: relation-specific in /macros/adapters/relations/<relation>, relation agnostic in /macros/relations ([#8449](https://github.com/dbt-labs/dbt-core/issues/8449))
-- Update typing to meet mypy standards ([#8396](https://github.com/dbt-labs/dbt-core/issues/8396))
-- Mypy errors - adapters/factory.py ([#8387](https://github.com/dbt-labs/dbt-core/issues/8387))
-- Added more type annotations. ([#8537](https://github.com/dbt-labs/dbt-core/issues/8537))
-- Audit potential circular dependencies ([#8349](https://github.com/dbt-labs/dbt-core/issues/8349))
-- Add functional test for advanced ref override ([#8566](https://github.com/dbt-labs/dbt-core/issues/8566))
-- Add typing to __init__ in base.py ([#8398](https://github.com/dbt-labs/dbt-core/issues/8398))
-- Fix untyped functions in task/runnable.py (mypy warning) ([#8402](https://github.com/dbt-labs/dbt-core/issues/8402))
-- add a test for ephemeral cte injection ([#8225](https://github.com/dbt-labs/dbt-core/issues/8225))
-- Fix test_numeric_values to look for more specific strings ([#8470](https://github.com/dbt-labs/dbt-core/issues/8470))
-- Pin types-requests<2.31.0 in `dev-requirements.txt` ([#8789](https://github.com/dbt-labs/dbt-core/issues/8789))
-- Add warning_tag to UnversionedBreakingChange ([#8827](https://github.com/dbt-labs/dbt-core/issues/8827))
-- Update v10 manifest schema to match 1.6 for testing schema compatibility ([#8835](https://github.com/dbt-labs/dbt-core/issues/8835))
-- Add a no-op runner for Saved Qeury ([#8893](https://github.com/dbt-labs/dbt-core/issues/8893))
-
-### Dependencies
-
-- Bump mypy from 1.3.0 to 1.4.0 ([#7912](https://github.com/dbt-labs/dbt-core/pull/7912))
-- Bump mypy from 1.4.0 to 1.4.1 ([#8219](https://github.com/dbt-labs/dbt-core/pull/8219))
-- Update pin for click<9 ([#8232](https://github.com/dbt-labs/dbt-core/pull/8232))
-- Add upper bound to sqlparse pin of <0.5 ([#8236](https://github.com/dbt-labs/dbt-core/pull/8236))
-- Support dbt-semantic-interfaces 0.2.0 ([#8250](https://github.com/dbt-labs/dbt-core/pull/8250))
-- Bump docker/build-push-action from 4 to 5 ([#8783](https://github.com/dbt-labs/dbt-core/pull/8783))
-- Upgrade dbt-semantic-interfaces dep to 0.3.0 ([#8819](https://github.com/dbt-labs/dbt-core/pull/8819))
-- Begin using DSI 0.4.x ([#8892](https://github.com/dbt-labs/dbt-core/pull/8892))
-
-### Contributors
-- [@anjutiwari](https://github.com/anjutiwari) ([#7428](https://github.com/dbt-labs/dbt-core/issues/7428), [#8223](https://github.com/dbt-labs/dbt-core/issues/8223))
-- [@benmosher](https://github.com/benmosher) ([#8480](https://github.com/dbt-labs/dbt-core/issues/8480))
-- [@d-kaneshiro](https://github.com/d-kaneshiro) ([#None](https://github.com/dbt-labs/dbt-core/issues/None))
-- [@dave-connors-3](https://github.com/dave-connors-3) ([#8153](https://github.com/dbt-labs/dbt-core/issues/8153), [#8589](https://github.com/dbt-labs/dbt-core/issues/8589))
-- [@dylan-murray](https://github.com/dylan-murray) ([#8683](https://github.com/dbt-labs/dbt-core/issues/8683))
-- [@ezraerb](https://github.com/ezraerb) ([#6154](https://github.com/dbt-labs/dbt-core/issues/6154))
-- [@gem7318](https://github.com/gem7318) ([#8010](https://github.com/dbt-labs/dbt-core/issues/8010))
-- [@jamezrin](https://github.com/jamezrin) ([#8211](https://github.com/dbt-labs/dbt-core/issues/8211))
-- [@jusbaldw](https://github.com/jusbaldw) ([#6643](https://github.com/dbt-labs/dbt-core/issues/6643))
-- [@lllong33](https://github.com/lllong33) ([#5301](https://github.com/dbt-labs/dbt-core/issues/5301))
-- [@marcodamore](https://github.com/marcodamore) ([#436](https://github.com/dbt-labs/dbt-core/issues/436))
-- [@mescanne](https://github.com/mescanne) ([#8614](https://github.com/dbt-labs/dbt-core/issues/8614))
-- [@pgoslatara](https://github.com/pgoslatara) ([#8100](https://github.com/dbt-labs/dbt-core/issues/8100))
-- [@philippeboyd](https://github.com/philippeboyd) ([#5374](https://github.com/dbt-labs/dbt-core/issues/5374))
-- [@ramonvermeulen](https://github.com/ramonvermeulen) ([#3990](https://github.com/dbt-labs/dbt-core/issues/3990))
-- [@renanleme](https://github.com/renanleme) ([#8692](https://github.com/dbt-labs/dbt-core/issues/8692))

.changes/1.7.1.md

@@ -1,8 +0,0 @@
-## dbt-core 1.7.1 - November 07, 2023
-
-### Fixes
-
-- Fix compilation exception running empty seed file and support new Integer agate data_type ([#8895](https://github.com/dbt-labs/dbt-core/issues/8895))
-- Update run_results.json from previous versions of dbt to support deferral and rerun from failure ([#9010](https://github.com/dbt-labs/dbt-core/issues/9010))
-- Use MANIFEST.in to recursively include all jinja templates; fixes issue where some templates were not included in the distribution ([#9016](https://github.com/dbt-labs/dbt-core/issues/9016))
-- Fix git repository with subdirectory for Deps ([#9000](https://github.com/dbt-labs/dbt-core/issues/9000))

.changes/1.7.10.md

@@ -1,11 +0,0 @@
-## dbt-core 1.7.10 - March 14, 2024
-
-### Fixes
-
-- Do not add duplicate input_measures ([#9360](https://github.com/dbt-labs/dbt-core/issues/9360))
-- Fix partial parsing `KeyError` on deleted schema files ([#8860](https://github.com/dbt-labs/dbt-core/issues/8860))
-- Support saved queries in `dbt list` ([#9532](https://github.com/dbt-labs/dbt-core/issues/9532))
-
-### Dependencies
-
-- Restrict protobuf to 4.* versions ([#9566](https://github.com/dbt-labs/dbt-core/pull/9566))

.changes/1.7.11.md

@@ -1,6 +0,0 @@
-## dbt-core 1.7.11 - March 28, 2024
-
-### Fixes
-
-- Tighten exception handling to avoid worker thread hangs. ([#9583](https://github.com/dbt-labs/dbt-core/issues/9583))
-- Add field wrapper to BaseRelation members that were missing it. ([#9681](https://github.com/dbt-labs/dbt-core/issues/9681))

.changes/1.7.12.md

@@ -1,6 +0,0 @@
-## dbt-core 1.7.12 - April 16, 2024
-
-### Fixes
-
-- Fix assorted source freshness edgecases so check is run or actionable information is given ([#9078](https://github.com/dbt-labs/dbt-core/issues/9078))
-- Exclude password-like fields for considering reparse ([#9795](https://github.com/dbt-labs/dbt-core/issues/9795))

.changes/1.7.13.md

@@ -1,8 +0,0 @@
-## dbt-core 1.7.13 - April 18, 2024
-
-### Security
-
-- Bump sqlparse to >=0.5.0, <0.6.0 to address GHSA-2m57-hf25-phgg ([#9951](https://github.com/dbt-labs/dbt-core/pull/9951))
-
-### Contributors
-- [@emmoop](https://github.com/emmoop) ([#9951](https://github.com/dbt-labs/dbt-core/pull/9951))

.changes/1.7.14.md

@@ -1,16 +0,0 @@
-## dbt-core 1.7.14 - May 02, 2024
-
-### Features
-
-- Move flags from UserConfig in profiles.yml to flags in dbt_project.yml ([#9183](https://github.com/dbt-labs/dbt-core/issues/9183))
-- Add require_explicit_package_overrides_for_builtin_materializations to dbt_project.yml flags, which can be used to opt-out of overriding built-in materializations from packages ([#10007](https://github.com/dbt-labs/dbt-core/issues/10007))
-
-### Fixes
-
-- remove materialized views from renambeable relation and remove a quote ([#127](https://github.com/dbt-labs/dbt-core/issues/127))
-- Replace usage of `Set` with `List` to fix issue with index updates intermittently happening out of order ([#72](https://github.com/dbt-labs/dbt-core/issues/72))
-
-### Under the Hood
-
-- Raise deprecation warning if installed package overrides built-in materialization ([#9971](https://github.com/dbt-labs/dbt-core/issues/9971))
-- Remove the final underscore from secret environment variable constants. ([#10052](https://github.com/dbt-labs/dbt-core/issues/10052))

.changes/1.7.15.md

@@ -1,9 +0,0 @@
-## dbt-core 1.7.15 - May 22, 2024
-
-### Fixes
-
-- Fix the semicolon semantics for indexes while respecting other bug fix ([#85](https://github.com/dbt-labs/dbt-core/issues/85))
-
-### Security
-
-- Explicitly bind to localhost in docs serve ([#10209](https://github.com/dbt-labs/dbt-core/issues/10209))

.changes/1.7.16.md

@@ -1,5 +0,0 @@
-## dbt-core 1.7.16 - June 05, 2024
-
-### Features
-
-- Add --host flag to dbt docs serve, defaulting to '127.0.0.1' ([#10229](https://github.com/dbt-labs/dbt-core/issues/10229))

.changes/1.7.17.md

@@ -1,5 +0,0 @@
-## dbt-core 1.7.17 - June 20, 2024
-
-### Docs
-
-- Fix npm security vulnerabilities as of June 2024 ([dbt-docs/#513](https://github.com/dbt-labs/dbt-docs/issues/513))

.changes/1.7.18.md

@@ -1,5 +0,0 @@
-## dbt-core 1.7.18 - August 07, 2024
-
-### Fixes
-
-- respect --quiet and --warn-error-options for flag deprecations ([#10105](https://github.com/dbt-labs/dbt-core/issues/10105))

.changes/1.7.2.md

@@ -1,16 +0,0 @@
-## dbt-core 1.7.2 - November 16, 2023
-
-### Features
-
-- Support setting export configs hierarchically via saved query and project configs ([#8956](https://github.com/dbt-labs/dbt-core/issues/8956))
-
-### Fixes
-
-- Fix formatting of tarball information in packages-lock.yml ([#9062](https://github.com/dbt-labs/dbt-core/issues/9062))
-
-### Under the Hood
-
-- Treat SystemExit as an interrupt if raised during node execution. ([#n/a](https://github.com/dbt-labs/dbt-core/issues/n/a))
-
-### Contributors
-- [@benmosher](https://github.com/benmosher) ([#n/a](https://github.com/dbt-labs/dbt-core/issues/n/a))

.changes/1.7.3.md

@@ -1,7 +0,0 @@
-## dbt-core 1.7.3 - November 29, 2023
-
-### Fixes
-
-- deps: Lock git packages to commit SHA during resolution ([#9050](https://github.com/dbt-labs/dbt-core/issues/9050))
-- deps: Use PackageRenderer to read package-lock.json ([#9127](https://github.com/dbt-labs/dbt-core/issues/9127))
-- Get sources working again in dbt docs generate ([#9119](https://github.com/dbt-labs/dbt-core/issues/9119))

.changes/1.7.4.md

@@ -1,12 +0,0 @@
-## dbt-core 1.7.4 - December 14, 2023
-
-### Features
-
-- Adds support for parsing conversion metric related properties for the semantic layer. ([#9203](https://github.com/dbt-labs/dbt-core/issues/9203))
-
-### Fixes
-
-- Ensure we produce valid jsonschema schemas for manifest, catalog, run-results, and sources ([#8991](https://github.com/dbt-labs/dbt-core/issues/8991))
-
-### Contributors
-- [@WilliamDee](https://github.com/WilliamDee) ([#9203](https://github.com/dbt-labs/dbt-core/issues/9203))

.changes/1.7.5.md

@@ -1,8 +0,0 @@
-## dbt-core 1.7.5 - January 18, 2024
-
-### Fixes
-
-- Preserve the value of vars and the --full-refresh flags when using retry. ([#9112](https://github.com/dbt-labs/dbt-core/issues/9112))
-
-### Contributors
-- [@peterallenwebb,](https://github.com/peterallenwebb,) ([#9112](https://github.com/dbt-labs/dbt-core/issues/9112))

.changes/1.7.6.md

@@ -1,6 +0,0 @@
-## dbt-core 1.7.6 - January 25, 2024
-
-### Fixes
-
-- Handle unknown `type_code` for model contracts ([#8877](https://github.com/dbt-labs/dbt-core/issues/8877), [#8353](https://github.com/dbt-labs/dbt-core/issues/8353))
-- Fix retry command run from CLI ([#9444](https://github.com/dbt-labs/dbt-core/issues/9444))

.changes/1.7.7.md

@@ -1,6 +0,0 @@
-## dbt-core 1.7.7 - February 01, 2024
-
-### Fixes
-
-- Fix seed and source selection in `dbt docs generate` ([#9161](https://github.com/dbt-labs/dbt-core/issues/9161))
-- Add TestGenerateCatalogWithExternalNodes, include empty nodes in node selection during docs generate ([#9456](https://github.com/dbt-labs/dbt-core/issues/9456))

.changes/1.7.8.md

@@ -1,6 +0,0 @@
-## dbt-core 1.7.8 - February 14, 2024
-
-### Fixes
-
-- When patching versioned models, set constraints after config ([#9364](https://github.com/dbt-labs/dbt-core/issues/9364))
-- Store node_info in node associated logging events ([#9557](https://github.com/dbt-labs/dbt-core/issues/9557))

.changes/1.7.9.md

@@ -1,18 +0,0 @@
-## dbt-core 1.7.9 - February 28, 2024
-
-### Fixes
-
-- Fix node_info contextvar handling so incorrect node_info doesn't persist ([#8866](https://github.com/dbt-labs/dbt-core/issues/8866))
-- Add target-path to retry ([#8948](https://github.com/dbt-labs/dbt-core/issues/8948))
-
-### Under the Hood
-
-- Make dbt-core compatible with Python 3.12 ([#9007](https://github.com/dbt-labs/dbt-core/issues/9007))
-- Restrict protobuf to major version 4. ([#9566](https://github.com/dbt-labs/dbt-core/issues/9566))
-
-### Security
-
-- Update Jinja2 to >= 3.1.3 to address CVE-2024-22195 ([#CVE-2024-22195](https://github.com/dbt-labs/dbt-core/pull/CVE-2024-22195))
-
-### Contributors
-- [@l1xnan](https://github.com/l1xnan) ([#9007](https://github.com/dbt-labs/dbt-core/issues/9007))

.changes/unreleased/Dependencies-20230621-005752.yaml

@@ -0,0 +1,6 @@
+kind: "Dependencies"
+body: "Bump mypy from 1.3.0 to 1.4.0"
+time: 2023-06-21T00:57:52.00000Z
+custom:
+  Author: dependabot[bot]
+  PR: 7912

.changes/unreleased/Docs-20230715-200907.yaml

@@ -0,0 +1,6 @@
+kind: Docs
+body: Corrected spelling of "Partiton"
+time: 2023-07-15T20:09:07.057361092+02:00
+custom:
+  Author: pgoslatara
+  Issue: "8100"

.changes/unreleased/Docs-20230718-192422.yaml

@@ -0,0 +1,6 @@
+kind: Docs
+body: Remove static SQL codeblock for metrics
+time: 2023-07-18T19:24:22.155323+02:00
+custom:
+  Author: marcodamore
+  Issue: "436"

.changes/unreleased/Fixes-20230625-142731.yaml

@@ -0,0 +1,6 @@
+kind: Fixes
+body: Fixed double-underline
+time: 2023-06-25T14:27:31.231253719+08:00
+custom:
+  Author: lllong33
+  Issue: "5301"

.changes/unreleased/Fixes-20230718-125518.yaml

@@ -0,0 +1,6 @@
+kind: Fixes
+body: Enable converting deprecation warnings to errors
+time: 2023-07-18T12:55:18.03914-04:00
+custom:
+  Author: michelleark
+  Issue: "8130"

.changes/unreleased/Fixes-20230720-122723.yaml

@@ -0,0 +1,6 @@
+kind: Fixes
+body: Add status to Parse Inline Error
+time: 2023-07-20T12:27:23.085084-07:00
+custom:
+  Author: ChenyuLInx
+  Issue: "8173"

.changes/unreleased/Fixes-20230720-161513.yaml

@@ -0,0 +1,6 @@
+kind: Fixes
+body: Ensure `warn_error_options` get serialized in `invocation_args_dict`
+time: 2023-07-20T16:15:13.761813-07:00
+custom:
+  Author: QMalcolm
+  Issue: "7694"

.changes/unreleased/Fixes-20230720-170112.yaml

@@ -0,0 +1,6 @@
+kind: Fixes
+body: Stop detecting materialization macros based on macro name
+time: 2023-07-20T17:01:12.496238-07:00
+custom:
+  Author: QMalcolm
+  Issue: "6231"

.changes/unreleased/Fixes-20230720-172422.yaml

@@ -0,0 +1,6 @@
+kind: Fixes
+body: Update `dbt deps` download retry logic to handle `EOFError` exceptions
+time: 2023-07-20T17:24:22.969951-07:00
+custom:
+  Author: QMalcolm
+  Issue: "6653"

.changes/unreleased/Fixes-20230726-104448.yaml

@@ -0,0 +1,6 @@
+kind: Fixes
+body: Improve handling of CTE injection with ephemeral models
+time: 2023-07-26T10:44:48.888451-04:00
+custom:
+  Author: gshank
+  Issue: "8213"

.changes/unreleased/Under the Hood-20230719-124611.yaml

@@ -0,0 +1,6 @@
+kind: Under the Hood
+body: Refactor flaky test pp_versioned_models
+time: 2023-07-19T12:46:11.972481-04:00
+custom:
+  Author: gshank
+  Issue: "7781"

.changes/unreleased/Under the Hood-20230719-163334.yaml

@@ -0,0 +1,6 @@
+kind: Under the Hood
+body: format exception from dbtPlugin.initialize
+time: 2023-07-19T16:33:34.586377-04:00
+custom:
+  Author: michelleark
+  Issue: "8152"

.changes/unreleased/Under the Hood-20230724-150654.yaml

@@ -0,0 +1,6 @@
+kind: Under the Hood
+body: A way to control maxBytes for a single dbt.log file
+time: 2023-07-24T15:06:54.263822-07:00
+custom:
+  Author: ChenyuLInx
+  Issue: "8199"

.changes/unreleased/Under the Hood-20230725-102609.yaml

@@ -0,0 +1,7 @@
+kind: Under the Hood
+body: Ref expressions with version can now be processed by the latest version of the
+  high-performance dbt-extractor library.
+time: 2023-07-25T10:26:09.902878-04:00
+custom:
+  Author: peterallenwebb
+  Issue: "7688"

.changes/unreleased/Under the Hood-20241016-144056.yaml

@@ -1,6 +0,0 @@
-kind: Under the Hood
-body: Remove support and testing for Python 3.8, which is now EOL.
-time: 2024-10-16T14:40:56.451972-04:00
-custom:
-  Author: gshank peterallenwebb
-  Issue: "10861"

.changie.yaml

@@ -31,7 +31,43 @@ kinds:
       - {{.Body}} ({{ range $index, $element := $IssueList }}{{if $index}}, {{end}}{{$element}}{{end}})
   - label: Under the Hood
   - label: Dependencies
+    changeFormat: |-
+      {{- $PRList := list }}
+      {{- $changes := splitList " " $.Custom.PR }}
+      {{- range $pullrequest := $changes }}
+        {{- $changeLink := "[#nbr](https://github.com/dbt-labs/dbt-core/pull/nbr)" | replace "nbr" $pullrequest }}
+        {{- $PRList = append $PRList $changeLink  }}
+      {{- end -}}
+      - {{.Body}} ({{ range $index, $element := $PRList }}{{if $index}}, {{end}}{{$element}}{{end}})
+    skipGlobalChoices: true
+    additionalChoices:
+      - key: Author
+        label: GitHub Username(s) (separated by a single space if multiple)
+        type: string
+        minLength: 3
+      - key: PR
+        label: GitHub Pull Request Number (separated by a single space if multiple)
+        type: string
+        minLength: 1
   - label: Security
+    changeFormat: |-
+      {{- $PRList := list }}
+      {{- $changes := splitList " " $.Custom.PR }}
+      {{- range $pullrequest := $changes }}
+        {{- $changeLink := "[#nbr](https://github.com/dbt-labs/dbt-core/pull/nbr)" | replace "nbr" $pullrequest }}
+        {{- $PRList = append $PRList $changeLink  }}
+      {{- end -}}
+      - {{.Body}} ({{ range $index, $element := $PRList }}{{if $index}}, {{end}}{{$element}}{{end}})
+    skipGlobalChoices: true
+    additionalChoices:
+      - key: Author
+        label: GitHub Username(s) (separated by a single space if multiple)
+        type: string
+        minLength: 3
+      - key: PR
+        label: GitHub Pull Request Number (separated by a single space if multiple)
+        type: string
+        minLength: 1
 
 newlines:
   afterChangelogHeader: 1
@@ -70,10 +106,18 @@ footerFormat: |
         {{- $changeList := splitList " " $change.Custom.Author }}
           {{- $IssueList := list }}
           {{- $changeLink := $change.Kind }}
-          {{- $changes := splitList " " $change.Custom.Issue }}
+          {{- if or (eq $change.Kind "Dependencies") (eq $change.Kind "Security") }}
-          {{- range $issueNbr := $changes }}
+            {{- $changes := splitList " " $change.Custom.PR }}
-            {{- $changeLink := "[#nbr](https://github.com/dbt-labs/dbt-core/issues/nbr)" | replace "nbr" $issueNbr }}
+            {{- range $issueNbr := $changes }}
-            {{- $IssueList = append $IssueList $changeLink  }}
+              {{- $changeLink := "[#nbr](https://github.com/dbt-labs/dbt-core/pull/nbr)" | replace "nbr" $issueNbr }}
+              {{- $IssueList = append $IssueList $changeLink  }}
+            {{- end -}}
+          {{- else }}
+            {{- $changes := splitList " " $change.Custom.Issue }}
+            {{- range $issueNbr := $changes }}
+              {{- $changeLink := "[#nbr](https://github.com/dbt-labs/dbt-core/issues/nbr)" | replace "nbr" $issueNbr }}
+              {{- $IssueList = append $IssueList $changeLink  }}
+            {{- end -}}
           {{- end }}
           {{- /* check if this contributor has other changes associated with them already */}}
           {{- if hasKey $contributorDict $author }}

.github/CODEOWNERS

@@ -13,6 +13,23 @@
 # the core team as a whole will be assigned
 *       @dbt-labs/core-team
 
+### OSS Tooling Guild
+
+/.github/                       @dbt-labs/guild-oss-tooling
+.bumpversion.cfg                @dbt-labs/guild-oss-tooling
+
+.changie.yaml                   @dbt-labs/guild-oss-tooling
+
+pre-commit-config.yaml          @dbt-labs/guild-oss-tooling
+pytest.ini                      @dbt-labs/guild-oss-tooling
+tox.ini                         @dbt-labs/guild-oss-tooling
+
+pyproject.toml                  @dbt-labs/guild-oss-tooling
+requirements.txt                @dbt-labs/guild-oss-tooling
+dev_requirements.txt            @dbt-labs/guild-oss-tooling
+/core/setup.py                  @dbt-labs/guild-oss-tooling
+/core/MANIFEST.in               @dbt-labs/guild-oss-tooling
+
 ### ADAPTERS
 
 # Adapter interface ("base" + "sql" adapter defaults, cache)
@@ -23,7 +40,7 @@
 
 # Postgres plugin
 /plugins/                       @dbt-labs/core-adapters
-/plugins/postgres/setup.py      @dbt-labs/core-adapters
+/plugins/postgres/setup.py      @dbt-labs/core-adapters @dbt-labs/guild-oss-tooling
 
 # Functional tests for adapter plugins
 /tests/adapter                  @dbt-labs/core-adapters

.github/ISSUE_TEMPLATE/implementation-ticket.yml

@@ -1,7 +1,7 @@
 name: 🛠️ Implementation
 description: This is an implementation ticket intended for use by the maintainers of dbt-core
 title: "[<project>] <title>"
-labels: ["user docs"]
+labels: ["user_docs"]
 body:
   - type: markdown
     attributes:
@@ -11,7 +11,7 @@ body:
       label: Housekeeping
       description: >
         A couple friendly reminders:
-          1. Remove the `user docs` label if the scope of this work does not require changes to https://docs.getdbt.com/docs: no end-user interface (e.g. yml spec, CLI, error messages, etc) or functional changes
+          1. Remove the `user_docs` label if the scope of this work does not require changes to https://docs.getdbt.com/docs: no end-user interface (e.g. yml spec, CLI, error messages, etc) or functional changes
           2. Link any blocking issues in the "Blocked on" field under the "Core devs & maintainers" project.
       options:
         - label: I am a maintainer of dbt-core
@@ -25,29 +25,11 @@ body:
       required: true
   - type: textarea
     attributes:
-      label: Acceptance criteria
+      label: Acceptance critera
       description: |
         What is the definition of done for this ticket? Include any relevant edge cases and/or test cases
     validations:
       required: true
-  - type: textarea
-    attributes:
-      label: Impact to Other Teams
-      description: |
-        Will this change impact other teams?  Include details of the kinds of changes required (new tests, code changes, related tickets) and _add the relevant `Impact:[team]` label_.
-      placeholder: |
-        Example: This change impacts `dbt-redshift` because the tests will need to be modified.  The `Impact:[Adapter]` label has been added.
-    validations:
-      required: true
-  - type: textarea
-    attributes:
-      label: Will backports be required?
-      description: |
-        Will this change need to be backported to previous versions?  Add details, possible blockers to backporting and _add the relevant backport labels `backport 1.x.latest`_
-      placeholder: |
-        Example: Backport to 1.6.latest, 1.5.latest and 1.4.latest.  Since 1.4 isn't using click, the backport may be complicated. The `backport 1.6.latest`, `backport 1.5.latest` and `backport 1.4.latest` labels have been added.
-    validations:
-      required: true
   - type: textarea
     attributes:
       label: Context

.github/actions/latest-wrangler/examples/example_workflow.yml

@@ -13,7 +13,7 @@ jobs:
   build:
     runs-on: ubuntu-latest
     steps:
-    - uses: actions/checkout@v4
+    - uses: actions/checkout@v3
     - name: Wrangle latest tag
       id: is_latest
       uses: ./.github/actions/latest-wrangler

.github/dependabot.yml

@@ -28,10 +28,3 @@ updates:
     schedule:
       interval: "weekly"
     rebase-strategy: "disabled"
-
-  # github dependencies
-  - package-ecosystem: "github-actions"
-    directory: "/"
-    schedule:
-      interval: "weekly"
-    rebase-strategy: "disabled"

.github/pull_request_template.md

@@ -1,12 +1,15 @@
-resolves #
+resolves #  
+[docs](https://github.com/dbt-labs/docs.getdbt.com/issues/new/choose) dbt-labs/docs.getdbt.com/#  
 
 <!---
   Include the number of the issue addressed by this PR above if applicable.
   PRs for code changes without an associated issue *will not be merged*.
   See CONTRIBUTING.md for more information.
 
-  Add the `user docs` label to this PR if it will need docs changes.  An 
+  Include the number of the docs issue that was opened for this PR. If
-  issue will get opened in docs.getdbt.com upon successful merge of this PR.
+  this change has no user-facing implications, "N/A" suffices instead. New
+  docs tickets can be created by clicking the link above or by going to
+  https://github.com/dbt-labs/docs.getdbt.com/issues/new/choose.
 -->
 
 ### Problem
@@ -30,4 +33,3 @@ resolves #
 - [ ] I have run this code in development and it appears to resolve the stated issue  
 - [ ] This PR includes tests, or tests are not required/relevant for this PR
 - [ ] This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX
-- [ ] This PR includes [type annotations](https://docs.python.org/3/library/typing.html) for new and modified functions

.github/workflows/bot-changelog.yml

@@ -58,4 +58,4 @@ jobs:
         commit_message: "Add automated changelog yaml from template for bot PR"
         changie_kind: ${{ matrix.changie_kind }}
         label: ${{ matrix.label }}
-        custom_changelog_string: "custom:\n  Author: ${{ github.event.pull_request.user.login }}\n  Issue: ${{ github.event.pull_request.number }}"
+        custom_changelog_string: "custom:\n  Author: ${{ github.event.pull_request.user.login }}\n  PR: ${{ github.event.pull_request.number }}"

.github/workflows/changelog-existence.yml

@@ -2,8 +2,10 @@
 # Checks that a file has been committed under the /.changes directory
 # as a new CHANGELOG entry.  Cannot check for a specific filename as
 # it is dynamically generated by change type and timestamp.
-# This workflow runs on pull_request_target because it requires
+# This workflow should not require any secrets since it runs for PRs
-# secrets to post comments.
+# from forked repos.
+# By default, secrets are not passed to workflows running from
+# a forked repo.
 
 # **why?**
 # Ensure code change gets reflected in the CHANGELOG.
@@ -17,7 +19,7 @@
 name: Check Changelog Entry
 
 on:
-  pull_request_target:
+  pull_request:
     types: [opened, reopened, labeled, unlabeled, synchronize]
   workflow_dispatch:
 

.github/workflows/docs-issue.yml

@@ -1,43 +0,0 @@
-# **what?**
-# Open an issue in docs.getdbt.com when a PR is labeled `user docs`
-
-# **why?**
-# To reduce barriers for keeping docs up to date
-
-# **when?**
-# When a PR is labeled `user docs` and is merged.  Runs on pull_request_target to run off the workflow already merged,
-# not the workflow that existed on the PR branch.  This allows old PRs to get comments.
-
-
-name: Open issues in docs.getdbt.com repo when a PR is labeled
-run-name: "Open an issue in docs.getdbt.com for PR #${{ github.event.pull_request.number }}"
-
-on:
-  pull_request_target:
-    types: [labeled, closed]
-
-defaults:
-  run:
-    shell: bash
-
-permissions:
-    issues: write # opens new issues
-    pull-requests: write # comments on PRs
-
-
-jobs:
-  open_issues:
-    # we only want to run this when the PR has been merged or the label in the labeled event is `user docs`.  Otherwise it runs the
-    # risk of duplicaton of issues being created due to merge and label both triggering this workflow to run and neither having
-    # generating the comment before the other runs.  This lives here instead of the shared workflow because this is where we
-    # decide if it should run or not.
-    if: |
-      (github.event.pull_request.merged == true) &&
-      ((github.event.action == 'closed' && contains( github.event.pull_request.labels.*.name, 'user docs')) ||
-      (github.event.action == 'labeled' && github.event.label.name == 'user docs'))
-    uses: dbt-labs/actions/.github/workflows/open-issue-in-repo.yml@main
-    with:
-        issue_repository: "dbt-labs/docs.getdbt.com"
-        issue_title: "Docs Changes Needed from ${{ github.event.repository.name }} PR #${{ github.event.pull_request.number }}"
-        issue_body: "At a minimum, update body to include a link to the page on docs.getdbt.com requiring updates and what part(s) of the page you would like to see updated."
-    secrets: inherit

.github/workflows/main.yml

@@ -36,7 +36,7 @@ defaults:
 # top-level adjustments can be made here
 env:
   # number of parallel processes to spawn for python integration testing
-  PYTHON_INTEGRATION_TEST_WORKERS: 5
+  PYTHON_INTEGRATION_TEST_WORKERS: ${{ vars.PYTHON_INTEGRATION_TEST_WORKERS }}
 
 jobs:
   code-quality:
@@ -47,12 +47,12 @@ jobs:
 
     steps:
       - name: Check out the repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@v3
 
       - name: Set up Python
-        uses: actions/setup-python@v5
+        uses: actions/setup-python@v4
         with:
-          python-version: '3.9'
+          python-version: '3.8'
 
       - name: Install python dependencies
         run: |
@@ -74,17 +74,17 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: [ "3.9", "3.10", "3.11", "3.12" ]
+        python-version: ["3.8", "3.9", "3.10", "3.11"]
 
     env:
       TOXENV: "unit"
 
     steps:
       - name: Check out the repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@v3
 
       - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v5
+        uses: actions/setup-python@v4
         with:
           python-version: ${{ matrix.python-version }}
 
@@ -95,12 +95,8 @@ jobs:
           python -m pip install tox
           tox --version
 
-      - name: Run unit tests
+      - name: Run tox
-        uses: nick-fields/retry@v3
+        run: tox
-        with:
-          timeout_minutes: 10
-          max_attempts: 3
-          command: tox -e unit
 
       - name: Get current date
         if: always()
@@ -111,10 +107,9 @@ jobs:
 
       - name: Upload Unit Test Coverage to Codecov
         if: ${{ matrix.python-version == '3.11' }}
-        uses: codecov/codecov-action@v4
+        uses: codecov/codecov-action@v3
-        with:
+        env:
-          token: ${{ secrets.CODECOV_TOKEN }}
+          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
-          flags: unit
 
   integration-metadata:
     name: integration test metadata generation
@@ -139,7 +134,7 @@ jobs:
       - name: generate include
         id: generate-include
         run: |
-          INCLUDE=('"python-version":"3.9","os":"windows-latest"' '"python-version":"3.9","os":"macos-12"' )
+          INCLUDE=('"python-version":"3.8","os":"windows-latest"' '"python-version":"3.8","os":"macos-latest"' )
           INCLUDE_GROUPS="["
           for include in ${INCLUDE[@]}; do
               for group in $(seq 1 ${{ env.PYTHON_INTEGRATION_TEST_WORKERS }}); do
@@ -161,7 +156,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: [ "3.9", "3.10", "3.11", "3.12" ]
+        python-version: ["3.8", "3.9", "3.10", "3.11"]
         os: [ubuntu-20.04]
         split-group: ${{ fromJson(needs.integration-metadata.outputs.split-groups) }}
         include: ${{ fromJson(needs.integration-metadata.outputs.include) }}
@@ -179,10 +174,10 @@ jobs:
 
     steps:
       - name: Check out the repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@v3
 
       - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v5
+        uses: actions/setup-python@v4
         with:
           python-version: ${{ matrix.python-version }}
 
@@ -205,12 +200,8 @@ jobs:
           python -m pip install tox
           tox --version
 
-      - name: Run integration tests
+      - name: Run tests
-        uses: nick-fields/retry@v3
+        run: tox -- --ddtrace
-        with:
-          timeout_minutes: 30
-          max_attempts: 3
-          command: tox -- --ddtrace
         env:
           PYTEST_ADDOPTS: ${{ format('--splits {0} --group {1}', env.PYTHON_INTEGRATION_TEST_WORKERS, matrix.split-group) }}
 
@@ -221,35 +212,26 @@ jobs:
           CURRENT_DATE=$(date +'%Y-%m-%dT%H_%M_%S') # no colons allowed for artifacts
           echo "date=$CURRENT_DATE" >> $GITHUB_OUTPUT
 
-      - uses: actions/upload-artifact@v4
+      - uses: actions/upload-artifact@v3
         if: always()
         with:
-          name: logs_${{ matrix.python-version }}_${{ matrix.os }}_${{ matrix.split-group }}_${{ steps.date.outputs.date }}
+          name: logs_${{ matrix.python-version }}_${{ matrix.os }}_${{ steps.date.outputs.date }}
           path: ./logs
 
       - name: Upload Integration Test Coverage to Codecov
         if: ${{ matrix.python-version == '3.11' }}
-        uses: codecov/codecov-action@v4
+        uses: codecov/codecov-action@v3
-        with:
+        env:
-          token: ${{ secrets.CODECOV_TOKEN }}
+          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
-          flags: integration
 
   integration-report:
-    if: ${{ always() }}
+    name: integration test suite
-    name: Integration Test Suite
     runs-on: ubuntu-latest
     needs: integration
     steps:
-      - name: "Integration Tests Failed"
+      - name: "[Notification] Integration test suite passes"
-        if: ${{ contains(needs.integration.result, 'failure') || contains(needs.integration.result, 'cancelled') }}
-        # when this is true the next step won't execute
         run: |
-          echo "::notice title='Integration test suite failed'"
+          echo "::notice title="Integration test suite passes""
-          exit 1
-
-      - name: "Integration Tests Passed"
-        run: |
-          echo "::notice title='Integration test suite passed'"
 
   build:
     name: build packages
@@ -258,12 +240,12 @@ jobs:
 
     steps:
       - name: Check out the repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@v3
 
       - name: Set up Python
-        uses: actions/setup-python@v5
+        uses: actions/setup-python@v4
         with:
-          python-version: '3.9'
+          python-version: '3.8'
 
       - name: Install python dependencies
         run: |
@@ -296,7 +278,7 @@ jobs:
       - name: Install source distributions
         # ignore dbt-1.0.0, which intentionally raises an error when installed from source
         run: |
-          find ./dist/*.gz -maxdepth 1 -type f | xargs python -m pip install --force-reinstall --find-links=dist/
+          find ./dist/dbt-[a-z]*.gz -maxdepth 1 -type f | xargs python -m pip install --force-reinstall --find-links=dist/
 
       - name: Check source distributions
         run: |

.github/workflows/model_performance.yml

@@ -48,7 +48,7 @@ jobs:
       # explicitly checkout the performance runner from main regardless of which
       # version we are modeling.
       - name: Checkout
-        uses: actions/checkout@v4
+        uses: actions/checkout@v3
         with:
           ref: main
 
@@ -87,12 +87,12 @@ jobs:
       # explicitly checkout the performance runner from main regardless of which
       # version we are modeling.
       - name: Checkout
-        uses: actions/checkout@v4
+        uses: actions/checkout@v3
         with:
           ref: main
 
       # attempts to access a previously cached runner
-      - uses: actions/cache@v4
+      - uses: actions/cache@v3
         id: cache
         with:
           path: ${{ env.RUNNER_CACHE_PATH }}
@@ -148,9 +148,9 @@ jobs:
           echo "release_branch: ${{ needs.set-variables.outputs.release_branch }}"
 
       - name: Setup Python
-        uses: actions/setup-python@v5
+        uses: actions/setup-python@v4
         with:
-          python-version: "3.9"
+          python-version: "3.8"
 
       - name: Install dbt
         run: pip install dbt-postgres==${{ needs.set-variables.outputs.release_id }}
@@ -160,13 +160,13 @@ jobs:
 
       # explicitly checkout main to get the latest project definitions
       - name: Checkout
-        uses: actions/checkout@v4
+        uses: actions/checkout@v3
         with:
           ref: main
 
       # this was built in the previous job so it will be there.
       - name: Fetch Runner
-        uses: actions/cache@v4
+        uses: actions/cache@v3
         id: cache
         with:
           path: ${{ env.RUNNER_CACHE_PATH }}
@@ -195,7 +195,7 @@ jobs:
       - name: '[DEBUG] ls baseline directory after run'
         run: ls -R performance/baselines/
 
-      - uses: actions/upload-artifact@v4
+      - uses: actions/upload-artifact@v3
         with:
           name: baseline
           path: performance/baselines/${{ needs.set-variables.outputs.release_id }}/
@@ -225,7 +225,7 @@ jobs:
           echo "release_branch: ${{ needs.set-variables.outputs.release_branch }}"
 
       - name: Checkout
-        uses: actions/checkout@v4
+        uses: actions/checkout@v3
         with:
           ref: ${{ matrix.base-branch }}
 
@@ -235,7 +235,7 @@ jobs:
           git push origin ${{ matrix.target-branch }}
           git branch --set-upstream-to=origin/${{ matrix.target-branch }} ${{ matrix.target-branch }}
 
-      - uses: actions/download-artifact@v4
+      - uses: actions/download-artifact@v3
         with:
           name: baseline
           path: performance/baselines/${{ needs.set-variables.outputs.release_id }}

.github/workflows/nightly-release.yml

@@ -39,7 +39,7 @@ jobs:
 
     steps:
       - name: "Checkout ${{ github.repository }} Branch ${{ env.RELEASE_BRANCH }}"
-        uses: actions/checkout@v4
+        uses: actions/checkout@v3
         with:
           ref: ${{ env.RELEASE_BRANCH }}
 

.github/workflows/release-docker.yml

@@ -36,7 +36,7 @@ jobs:
       latest: ${{ steps.latest.outputs.latest }}
       minor_latest: ${{ steps.latest.outputs.minor_latest }}
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@v3
       - name: Split version
         id: version
         run: |
@@ -83,7 +83,7 @@ jobs:
           password: ${{ secrets.GITHUB_TOKEN }}
 
       - name: Build and push MAJOR.MINOR.PATCH tag
-        uses: docker/build-push-action@v5
+        uses: docker/build-push-action@v4
         with:
           file: docker/Dockerfile
           push: True
@@ -94,7 +94,7 @@ jobs:
             ghcr.io/dbt-labs/${{ github.event.inputs.package }}:${{ github.event.inputs.version_number }}
 
       - name: Build and push MINOR.latest tag
-        uses: docker/build-push-action@v5
+        uses: docker/build-push-action@v4
         if: ${{ needs.get_version_meta.outputs.minor_latest == 'True' }}
         with:
           file: docker/Dockerfile
@@ -106,7 +106,7 @@ jobs:
             ghcr.io/dbt-labs/${{ github.event.inputs.package }}:${{ needs.get_version_meta.outputs.major }}.${{ needs.get_version_meta.outputs.minor }}.latest
 
       - name: Build and push latest tag
-        uses: docker/build-push-action@v5
+        uses: docker/build-push-action@v4
         if: ${{ needs.get_version_meta.outputs.latest == 'True' }}
         with:
           file: docker/Dockerfile

.github/workflows/repository-cleanup.yml

@@ -1,30 +0,0 @@
-# **what?**
-# Cleanup branches left over from automation and testing.  Also cleanup
-# draft releases from release testing.
-
-# **why?**
-# The automations are leaving behind branches and releases that clutter
-# the repository.  Sometimes we need them to debug processes so we don't
-# want them immediately deleted.  Running on Saturday to avoid running
-# at the same time as an actual release to prevent breaking a release
-# mid-release.
-
-# **when?**
-# Mainly on a schedule of 12:00 Saturday.
-# Manual trigger can also run on demand
-
-name: Repository Cleanup
-
-on:
-  schedule:
-    - cron: '0 12 * * SAT' # At 12:00 on Saturday - details in `why` above
-
-  workflow_dispatch: # for manual triggering
-
-permissions:
-  contents: write
-
-jobs:
-  cleanup-repo:
-    uses: dbt-labs/actions/.github/workflows/repository-cleanup.yml@main
-    secrets: inherit

.github/workflows/schema-check.yml

@@ -37,20 +37,21 @@ jobs:
 
     steps:
       - name: Set up Python
-        uses: actions/setup-python@v5
+        uses: actions/setup-python@v4
         with:
-          python-version: 3.9
+          python-version: 3.8
 
       - name: Checkout dbt repo
-        uses: actions/checkout@v4
+        uses: actions/checkout@v3
         with:
             path: ${{ env.DBT_REPO_DIRECTORY }}
 
       - name: Checkout schemas.getdbt.com repo
-        uses: actions/checkout@v4
+        uses: actions/checkout@v3
         with:
           repository: dbt-labs/schemas.getdbt.com
           ref: 'main'
+          ssh-key: ${{ secrets.SCHEMA_SSH_PRIVATE_KEY }}
           path: ${{ env.SCHEMA_REPO_DIRECTORY }}
 
       - name: Generate current schema
@@ -62,14 +63,27 @@ jobs:
           pip install -r dev-requirements.txt -r editable-requirements.txt
           python scripts/collect-artifact-schema.py --path ${{ env.LATEST_SCHEMA_PATH }}
 
+      # Copy generated schema files into the schemas.getdbt.com repo
+      # Do a git diff to find any changes
+      # Ignore any date or version changes though
       - name: Compare schemas
         run: |
           cp -r ${{ env.LATEST_SCHEMA_PATH }}/dbt ${{ env.SCHEMA_REPO_DIRECTORY }}
           cd ${{ env.SCHEMA_REPO_DIRECTORY }}
-          git diff -I='*[0-9]{4}-[0-9]{2}-[0-9]{2}' -I='*[0-9]+\.[0-9]+\.[0-9]+' --exit-code > ${{ env.SCHEMA_DIFF_ARTIFACT }}
+          diff_results=$(git diff -I='*[0-9]{4}-(0[1-9]|1[0-2])-(0[1-9]|[1-2][0-9]|3[0-1])T' \
+          -I='*[0-9]{1}.[0-9]{2}.[0-9]{1}(rc[0-9]|b[0-9]| )' --compact-summary)
+          if [[ $(echo diff_results) ]]; then
+            echo $diff_results
+            echo "Schema changes detected!"
+            git diff -I='*[0-9]{4}-(0[1-9]|1[0-2])-(0[1-9]|[1-2][0-9]|3[0-1])T' \
+              -I='*[0-9]{1}.[0-9]{2}.[0-9]{1}(rc[0-9]|b[0-9]| )' > ${{ env.SCHEMA_DIFF_ARTIFACT }}
+            exit 1
+          else
+            echo "No schema changes detected"
+          fi
 
       - name: Upload schema diff
-        uses: actions/upload-artifact@v4
+        uses: actions/upload-artifact@v3
         if: ${{ failure() }}
         with:
           name: 'schema_schanges.txt'

.github/workflows/structured-logging-schema-check.yml

@@ -21,7 +21,7 @@ permissions: read-all
 # top-level adjustments can be made here
 env:
   # number of parallel processes to spawn for python testing
-  PYTHON_INTEGRATION_TEST_WORKERS: 5
+  PYTHON_INTEGRATION_TEST_WORKERS: ${{ vars.PYTHON_INTEGRATION_TEST_WORKERS }}
 
 jobs:
   integration-metadata:
@@ -69,14 +69,14 @@ jobs:
 
     steps:
       - name: checkout dev
-        uses: actions/checkout@v4
+        uses: actions/checkout@v3
         with:
           persist-credentials: false
 
       - name: Setup Python
-        uses: actions/setup-python@v5
+        uses: actions/setup-python@v4
         with:
-          python-version: "3.9"
+          python-version: "3.8"
 
       - name: Install python dependencies
         run: |
@@ -94,11 +94,7 @@ jobs:
       # integration tests generate a ton of logs in different files. the next step will find them all.
       # we actually care if these pass, because the normal test run doesn't usually include many json log outputs
       - name: Run integration tests
-        uses: nick-fields/retry@v3
+        run: tox -e integration -- -nauto
-        with:
-          timeout_minutes: 30
-          max_attempts: 3
-          command: tox -e integration -- -nauto
         env:
           PYTEST_ADDOPTS: ${{ format('--splits {0} --group {1}', env.PYTHON_INTEGRATION_TEST_WORKERS, matrix.split-group) }}
 

.github/workflows/test-repeater.yml

@@ -27,6 +27,7 @@ on:
         description: 'Version of Python to Test Against'
         type: choice
         options:
+          - '3.8'
           - '3.9'
           - '3.10'
           - '3.11'
@@ -35,7 +36,7 @@ on:
         type: choice
         options:
           - 'ubuntu-latest'
-          - 'macos-12'
+          - 'macos-latest'
           - 'windows-latest'
       num_runs_per_batch:
         description: 'Max number of times to run the test per batch.  We always run 10 batches.'
@@ -82,12 +83,12 @@ jobs:
 
     steps:
       - name: "Checkout code"
-        uses: actions/checkout@v4
+        uses: actions/checkout@v3
         with:
           ref: ${{ inputs.branch }}
 
       - name: "Setup Python"
-        uses: actions/setup-python@v5
+        uses: actions/setup-python@v4
         with:
           python-version: "${{ inputs.python_version }}"
 
@@ -100,7 +101,7 @@ jobs:
 
         # mac and windows don't use make due to limitations with docker with those runners in GitHub
       - name: "Set up postgres (macos)"
-        if: inputs.os == 'macos-12'
+        if: inputs.os == 'macos-latest'
         uses: ./.github/actions/setup-postgres-macos
 
       - name: "Set up postgres (windows)"

.pre-commit-config.yaml

@@ -3,7 +3,7 @@
 
 exclude: ^(core/dbt/docs/build/|core/dbt/events/types_pb2.py)
 
-# Force all unspecified python hooks to run python 3.9
+# Force all unspecified python hooks to run python 3.8
 default_language_version:
   python: python3
 
@@ -37,7 +37,7 @@ repos:
     alias: flake8-check
     stages: [manual]
 - repo: https://github.com/pre-commit/mirrors-mypy
-  rev: v1.4.1
+  rev: v1.4.0
   hooks:
   - id: mypy
     # N.B.: Mypy is... a bit fragile.

CHANGELOG.md

@@ -5,342 +5,6 @@
 - "Breaking changes" listed under a version may require action from end users or external maintainers when upgrading to that version.
 - Do not edit this file directly. This file is auto-generated using [changie](https://github.com/miniscruff/changie). For details on how to document a change, see [the contributing guide](https://github.com/dbt-labs/dbt-core/blob/main/CONTRIBUTING.md#adding-changelog-entry)
 
-## dbt-core 1.7.18 - August 07, 2024
-
-### Fixes
-
-- respect --quiet and --warn-error-options for flag deprecations ([#10105](https://github.com/dbt-labs/dbt-core/issues/10105))
-
-
-
-## dbt-core 1.7.17 - June 20, 2024
-
-### Docs
-
-- Fix npm security vulnerabilities as of June 2024 ([dbt-docs/#513](https://github.com/dbt-labs/dbt-docs/issues/513))
-
-## dbt-core 1.7.16 - June 05, 2024
-
-### Features
-
-- Add --host flag to dbt docs serve, defaulting to '127.0.0.1' ([#10229](https://github.com/dbt-labs/dbt-core/issues/10229))
-
-## dbt-core 1.7.15 - May 22, 2024
-
-### Fixes
-
-- Fix the semicolon semantics for indexes while respecting other bug fix ([#85](https://github.com/dbt-labs/dbt-core/issues/85))
-
-### Security
-
-- Explicitly bind to localhost in docs serve ([#10209](https://github.com/dbt-labs/dbt-core/issues/10209))
-
-## dbt-core 1.7.14 - May 02, 2024
-
-### Features
-
-- Move flags from UserConfig in profiles.yml to flags in dbt_project.yml ([#9183](https://github.com/dbt-labs/dbt-core/issues/9183))
-- Add require_explicit_package_overrides_for_builtin_materializations to dbt_project.yml flags, which can be used to opt-out of overriding built-in materializations from packages ([#10007](https://github.com/dbt-labs/dbt-core/issues/10007))
-
-### Fixes
-
-- remove materialized views from renambeable relation and remove a quote ([#127](https://github.com/dbt-labs/dbt-core/issues/127))
-- Replace usage of `Set` with `List` to fix issue with index updates intermittently happening out of order ([#72](https://github.com/dbt-labs/dbt-core/issues/72))
-
-### Under the Hood
-
-- Raise deprecation warning if installed package overrides built-in materialization ([#9971](https://github.com/dbt-labs/dbt-core/issues/9971))
-- Remove the final underscore from secret environment variable constants. ([#10052](https://github.com/dbt-labs/dbt-core/issues/10052))
-
-## dbt-core 1.7.13 - April 18, 2024
-
-### Security
-
-- Bump sqlparse to >=0.5.0, <0.6.0 to address GHSA-2m57-hf25-phgg ([#9951](https://github.com/dbt-labs/dbt-core/pull/9951))
-
-### Contributors
-- [@emmoop](https://github.com/emmoop) ([#9951](https://github.com/dbt-labs/dbt-core/pull/9951))
-
-## dbt-core 1.7.12 - April 16, 2024
-
-### Fixes
-
-- Fix assorted source freshness edgecases so check is run or actionable information is given ([#9078](https://github.com/dbt-labs/dbt-core/issues/9078))
-- Exclude password-like fields for considering reparse ([#9795](https://github.com/dbt-labs/dbt-core/issues/9795))
-
-## dbt-core 1.7.11 - March 28, 2024
-
-### Fixes
-
-- Tighten exception handling to avoid worker thread hangs. ([#9583](https://github.com/dbt-labs/dbt-core/issues/9583))
-- Add field wrapper to BaseRelation members that were missing it. ([#9681](https://github.com/dbt-labs/dbt-core/issues/9681))
-
-## dbt-core 1.7.10 - March 14, 2024
-
-### Fixes
-
-- Do not add duplicate input_measures ([#9360](https://github.com/dbt-labs/dbt-core/issues/9360))
-- Fix partial parsing `KeyError` on deleted schema files ([#8860](https://github.com/dbt-labs/dbt-core/issues/8860))
-- Support saved queries in `dbt list` ([#9532](https://github.com/dbt-labs/dbt-core/issues/9532))
-
-### Dependencies
-
-- Restrict protobuf to 4.* versions ([#9566](https://github.com/dbt-labs/dbt-core/pull/9566))
-
-## dbt-core 1.7.9 - February 28, 2024
-
-### Fixes
-
-- Fix node_info contextvar handling so incorrect node_info doesn't persist ([#8866](https://github.com/dbt-labs/dbt-core/issues/8866))
-- Add target-path to retry ([#8948](https://github.com/dbt-labs/dbt-core/issues/8948))
-
-### Under the Hood
-
-- Make dbt-core compatible with Python 3.12 ([#9007](https://github.com/dbt-labs/dbt-core/issues/9007))
-- Restrict protobuf to major version 4. ([#9566](https://github.com/dbt-labs/dbt-core/issues/9566))
-
-### Security
-
-- Update Jinja2 to >= 3.1.3 to address CVE-2024-22195 ([#CVE-2024-22195](https://github.com/dbt-labs/dbt-core/pull/CVE-2024-22195))
-
-### Contributors
-- [@l1xnan](https://github.com/l1xnan) ([#9007](https://github.com/dbt-labs/dbt-core/issues/9007))
-
-## dbt-core 1.7.8 - February 14, 2024
-
-### Fixes
-
-- When patching versioned models, set constraints after config ([#9364](https://github.com/dbt-labs/dbt-core/issues/9364))
-- Store node_info in node associated logging events ([#9557](https://github.com/dbt-labs/dbt-core/issues/9557))
-
-## dbt-core 1.7.7 - February 01, 2024
-
-### Fixes
-
-- Fix seed and source selection in `dbt docs generate` ([#9161](https://github.com/dbt-labs/dbt-core/issues/9161))
-- Add TestGenerateCatalogWithExternalNodes, include empty nodes in node selection during docs generate ([#9456](https://github.com/dbt-labs/dbt-core/issues/9456))
-
-## dbt-core 1.7.6 - January 25, 2024
-
-### Fixes
-
-- Handle unknown `type_code` for model contracts ([#8877](https://github.com/dbt-labs/dbt-core/issues/8877), [#8353](https://github.com/dbt-labs/dbt-core/issues/8353))
-- Fix retry command run from CLI ([#9444](https://github.com/dbt-labs/dbt-core/issues/9444))
-
-## dbt-core 1.7.5 - January 18, 2024
-
-### Fixes
-
-- Preserve the value of vars and the --full-refresh flags when using retry. ([#9112](https://github.com/dbt-labs/dbt-core/issues/9112))
-
-### Contributors
-- [@peterallenwebb,](https://github.com/peterallenwebb,) ([#9112](https://github.com/dbt-labs/dbt-core/issues/9112))
-
-## dbt-core 1.7.4 - December 14, 2023
-
-### Features
-
-- Adds support for parsing conversion metric related properties for the semantic layer. ([#9203](https://github.com/dbt-labs/dbt-core/issues/9203))
-
-### Fixes
-
-- Ensure we produce valid jsonschema schemas for manifest, catalog, run-results, and sources ([#8991](https://github.com/dbt-labs/dbt-core/issues/8991))
-
-### Contributors
-- [@WilliamDee](https://github.com/WilliamDee) ([#9203](https://github.com/dbt-labs/dbt-core/issues/9203))
-
-## dbt-core 1.7.3 - November 29, 2023
-
-### Fixes
-
-- deps: Lock git packages to commit SHA during resolution ([#9050](https://github.com/dbt-labs/dbt-core/issues/9050))
-- deps: Use PackageRenderer to read package-lock.json ([#9127](https://github.com/dbt-labs/dbt-core/issues/9127))
-- Get sources working again in dbt docs generate ([#9119](https://github.com/dbt-labs/dbt-core/issues/9119))
-
-## dbt-core 1.7.2 - November 16, 2023
-
-### Features
-
-- Support setting export configs hierarchically via saved query and project configs ([#8956](https://github.com/dbt-labs/dbt-core/issues/8956))
-
-### Fixes
-
-- Fix formatting of tarball information in packages-lock.yml ([#9062](https://github.com/dbt-labs/dbt-core/issues/9062))
-
-### Under the Hood
-
-- Treat SystemExit as an interrupt if raised during node execution. ([#n/a](https://github.com/dbt-labs/dbt-core/issues/n/a))
-
-### Contributors
-- [@benmosher](https://github.com/benmosher) ([#n/a](https://github.com/dbt-labs/dbt-core/issues/n/a))
-
-## dbt-core 1.7.1 - November 07, 2023
-
-### Fixes
-
-- Fix compilation exception running empty seed file and support new Integer agate data_type ([#8895](https://github.com/dbt-labs/dbt-core/issues/8895))
-- Update run_results.json from previous versions of dbt to support deferral and rerun from failure ([#9010](https://github.com/dbt-labs/dbt-core/issues/9010))
-- Use MANIFEST.in to recursively include all jinja templates; fixes issue where some templates were not included in the distribution ([#9016](https://github.com/dbt-labs/dbt-core/issues/9016))
-- Fix git repository with subdirectory for Deps ([#9000](https://github.com/dbt-labs/dbt-core/issues/9000))
-
-## dbt-core 1.7.0 - November 02, 2023
-
-### Breaking Changes
-
-- Removed the FirstRunResultError and AfterFirstRunResultError event types, using the existing RunResultError in their place. ([#7963](https://github.com/dbt-labs/dbt-core/issues/7963))
-
-### Features
-
-- add log file of installed packages via dbt deps ([#6643](https://github.com/dbt-labs/dbt-core/issues/6643))
-- Enable re-population of metadata vars post-environment change during programmatic invocation ([#8010](https://github.com/dbt-labs/dbt-core/issues/8010))
-- Added support to configure a delimiter for a seed file, defaults to comma ([#3990](https://github.com/dbt-labs/dbt-core/issues/3990))
-- Allow specification of `create_metric: true` on measures ([#8125](https://github.com/dbt-labs/dbt-core/issues/8125))
-- Add node attributes related to compilation to run_results.json ([#7519](https://github.com/dbt-labs/dbt-core/issues/7519))
-- Add --no-inject-ephemeral-ctes flag for `compile` command, for usage by linting. ([#8480](https://github.com/dbt-labs/dbt-core/issues/8480))
-- Support configuration of semantic models with the addition of enable/disable and group enablement. ([#7968](https://github.com/dbt-labs/dbt-core/issues/7968))
-- Accept a `dbt-cloud` config in dbt_project.yml ([#8438](https://github.com/dbt-labs/dbt-core/issues/8438))
-- Support atomic replace in the global replace macro ([#8539](https://github.com/dbt-labs/dbt-core/issues/8539))
-- Use translate_type on data_type in model.columns in templates by default, remove no op `TYPE_LABELS` ([#8007](https://github.com/dbt-labs/dbt-core/issues/8007))
-- Add an option to generate static documentation ([#8614](https://github.com/dbt-labs/dbt-core/issues/8614))
-- Allow setting "access" as a config in addition to as a property ([#8383](https://github.com/dbt-labs/dbt-core/issues/8383))
-- Loosen typing requirement on renameable/replaceable relations to Iterable to allow adapters more flexibility in registering relation types, include docstrings as suggestions ([#8647](https://github.com/dbt-labs/dbt-core/issues/8647))
-- Add support for optional label in semantic_models, measures, dimensions and entities. ([#8595](https://github.com/dbt-labs/dbt-core/issues/8595), [#8755](https://github.com/dbt-labs/dbt-core/issues/8755))
-- Allow adapters to include package logs in dbt standard logging  ([#7859](https://github.com/dbt-labs/dbt-core/issues/7859))
-- Support storing test failures as views ([#6914](https://github.com/dbt-labs/dbt-core/issues/6914))
-- resolve packages with same git repo and unique subdirectory ([#5374](https://github.com/dbt-labs/dbt-core/issues/5374))
-- Add new ResourceReport event to record memory/cpu/io metrics ([#8342](https://github.com/dbt-labs/dbt-core/issues/8342))
-- Adding `date_spine` macro (and supporting macros) from dbt-utils to dbt-core ([#8172](https://github.com/dbt-labs/dbt-core/issues/8172))
-- Support `fill_nulls_with` and `join_to_timespine` for metric nodes ([#8593](https://github.com/dbt-labs/dbt-core/issues/8593), [#8755](https://github.com/dbt-labs/dbt-core/issues/8755))
-- Raise a warning when a contracted model has a numeric field without scale defined ([#8183](https://github.com/dbt-labs/dbt-core/issues/8183))
-- Added support for retrieving partial catalog information from a schema ([#8521](https://github.com/dbt-labs/dbt-core/issues/8521))
-- Add meta attribute to SemanticModels config ([#8511](https://github.com/dbt-labs/dbt-core/issues/8511))
-- Selectors with docs generate limits catalog generation ([#6014](https://github.com/dbt-labs/dbt-core/issues/6014))
-- Allow freshness to be determined via DBMS metadata for supported adapters ([#8704](https://github.com/dbt-labs/dbt-core/issues/8704))
-- Add support semantic layer SavedQuery node type ([#8594](https://github.com/dbt-labs/dbt-core/issues/8594))
-- Add exports to SavedQuery spec ([#8892](https://github.com/dbt-labs/dbt-core/issues/8892))
-
-### Fixes
-
-- Copy dir during `dbt deps` if symlink fails ([#7428](https://github.com/dbt-labs/dbt-core/issues/7428), [#8223](https://github.com/dbt-labs/dbt-core/issues/8223))
-- If --profile specified with dbt-init, create the project with the specified profile ([#6154](https://github.com/dbt-labs/dbt-core/issues/6154))
-- Fixed double-underline ([#5301](https://github.com/dbt-labs/dbt-core/issues/5301))
-- Copy target_schema from config into snapshot node ([#6745](https://github.com/dbt-labs/dbt-core/issues/6745))
-- Enable converting deprecation warnings to errors ([#8130](https://github.com/dbt-labs/dbt-core/issues/8130))
-- Add status to Parse Inline Error ([#8173](https://github.com/dbt-labs/dbt-core/issues/8173))
-- Ensure `warn_error_options` get serialized in `invocation_args_dict` ([#7694](https://github.com/dbt-labs/dbt-core/issues/7694))
-- Stop detecting materialization macros based on macro name ([#6231](https://github.com/dbt-labs/dbt-core/issues/6231))
-- Update `dbt deps` download retry logic to handle `EOFError` exceptions ([#6653](https://github.com/dbt-labs/dbt-core/issues/6653))
-- Improve handling of CTE injection with ephemeral models ([#8213](https://github.com/dbt-labs/dbt-core/issues/8213))
-- Fix unbound local variable error in `checked_agg_time_dimension_for_measure` ([#8230](https://github.com/dbt-labs/dbt-core/issues/8230))
-- Ensure runtime errors are raised for graph runnable tasks (compile, show, run, etc) ([#8166](https://github.com/dbt-labs/dbt-core/issues/8166))
-- Fix retry not working with log-file-max-bytes ([#8297](https://github.com/dbt-labs/dbt-core/issues/8297))
-- Add explicit support for integers for the show command ([#8153](https://github.com/dbt-labs/dbt-core/issues/8153))
-- Detect changes to model access, version, or latest_version in state:modified ([#8189](https://github.com/dbt-labs/dbt-core/issues/8189))
-- Add connection status into list of statuses for dbt debug ([#8350](https://github.com/dbt-labs/dbt-core/issues/8350))
-- fix fqn-selection for external versioned models ([#8374](https://github.com/dbt-labs/dbt-core/issues/8374))
-- Fix: DbtInternalError after model that previously ref'd external model is deleted ([#8375](https://github.com/dbt-labs/dbt-core/issues/8375))
-- Fix using list command with path selector and project-dir ([#8385](https://github.com/dbt-labs/dbt-core/issues/8385))
-- Remedy performance regression by only writing run_results.json once. ([#8360](https://github.com/dbt-labs/dbt-core/issues/8360))
-- Add support for swapping materialized views with tables/views and vice versa ([#8449](https://github.com/dbt-labs/dbt-core/issues/8449))
-- Turn breaking changes to contracted models into warnings for unversioned models ([#8384](https://github.com/dbt-labs/dbt-core/issues/8384), [#8282](https://github.com/dbt-labs/dbt-core/issues/8282))
-- Ensure parsing does not break when `window_groupings` is not specified for `non_additive_dimension` ([#8453](https://github.com/dbt-labs/dbt-core/issues/8453))
-- fix ambiguous reference error for tests and versions when model name is duplicated across packages ([#8327](https://github.com/dbt-labs/dbt-core/issues/8327), [#8493](https://github.com/dbt-labs/dbt-core/issues/8493))
-- Fix "Internal Error: Expected node <unique-id> not found in manifest" when depends_on set on ModelNodeArgs ([#8506](https://github.com/dbt-labs/dbt-core/issues/8506))
-- Fix snapshot success message ([#7583](https://github.com/dbt-labs/dbt-core/issues/7583))
-- Parse the correct schema version from manifest ([#8544](https://github.com/dbt-labs/dbt-core/issues/8544))
-- make version comparison insensitive to order ([#8571](https://github.com/dbt-labs/dbt-core/issues/8571))
-- Update metric helper functions to work with new semantic layer metrics ([#8134](https://github.com/dbt-labs/dbt-core/issues/8134))
-- Disallow cleaning paths outside current working directory ([#8318](https://github.com/dbt-labs/dbt-core/issues/8318))
-- Warn when --state == --target ([#8160](https://github.com/dbt-labs/dbt-core/issues/8160))
-- update dbt show to include limit in DWH query  ([#8496,](https://github.com/dbt-labs/dbt-core/issues/8496,), [#8417](https://github.com/dbt-labs/dbt-core/issues/8417))
-- Support quoted parameter list for MultiOption CLI options. ([#8598](https://github.com/dbt-labs/dbt-core/issues/8598))
-- Support global flags passed in after subcommands ([#6497](https://github.com/dbt-labs/dbt-core/issues/6497))
-- Lower bound of `8.0.2` for `click` ([#8683](https://github.com/dbt-labs/dbt-core/issues/8683))
-- Fixes test type edges filter ([#8692](https://github.com/dbt-labs/dbt-core/issues/8692))
-- semantic models in graph selection ([#8589](https://github.com/dbt-labs/dbt-core/issues/8589))
-- Support doc blocks in nested semantic model YAML ([#8509](https://github.com/dbt-labs/dbt-core/issues/8509))
-- avoid double-rendering sql_header in dbt show ([#8739](https://github.com/dbt-labs/dbt-core/issues/8739))
-- Fix tag selection for projects with semantic models ([#8749](https://github.com/dbt-labs/dbt-core/issues/8749))
-- Foreign key constraint on incremental model results in Database Error ([#8022](https://github.com/dbt-labs/dbt-core/issues/8022))
-- Support docs blocks on versioned model column descriptions ([#8540](https://github.com/dbt-labs/dbt-core/issues/8540))
-- Enable seeds to be handled from stored manifest data ([#6875](https://github.com/dbt-labs/dbt-core/issues/6875))
-- Override path-like args in dbt retry ([#8682](https://github.com/dbt-labs/dbt-core/issues/8682))
-- Group updates on unmodified nodes are handled gracefully for state:modified ([#8371](https://github.com/dbt-labs/dbt-core/issues/8371))
-- Partial parsing fix for adding groups and updating models at the same time ([#8697](https://github.com/dbt-labs/dbt-core/issues/8697))
-- Fix partial parsing not working for semantic model change ([#8859](https://github.com/dbt-labs/dbt-core/issues/8859))
-- Rework get_catalog implementation to retain previous adapter interface semantics ([#8846](https://github.com/dbt-labs/dbt-core/issues/8846))
-- Add back contract enforcement for temporary tables on postgres ([#8857](https://github.com/dbt-labs/dbt-core/issues/8857))
-- Add version to fqn when version==0 ([#8836](https://github.com/dbt-labs/dbt-core/issues/8836))
-- Fix cased comparison in catalog-retrieval function. ([#8939](https://github.com/dbt-labs/dbt-core/issues/8939))
-- Catalog queries now assign the correct type to materialized views ([#8864](https://github.com/dbt-labs/dbt-core/issues/8864))
-- Make relation filtering None-tolerant for maximal flexibility across adapters. ([#8974](https://github.com/dbt-labs/dbt-core/issues/8974))
-
-### Docs
-
-- Corrected spelling of "Partiton" ([dbt-docs/#8100](https://github.com/dbt-labs/dbt-docs/issues/8100))
-- Remove static SQL codeblock for metrics ([dbt-docs/#436](https://github.com/dbt-labs/dbt-docs/issues/436))
-- fixed comment util.py ([dbt-docs/#None](https://github.com/dbt-labs/dbt-docs/issues/None))
-- Fix newline escapes and improve formatting in docker README ([dbt-docs/#8211](https://github.com/dbt-labs/dbt-docs/issues/8211))
-- Display contract and column constraints on the model page ([dbt-docs/#433](https://github.com/dbt-labs/dbt-docs/issues/433))
-- Display semantic model details in docs ([dbt-docs/#431](https://github.com/dbt-labs/dbt-docs/issues/431))
-
-### Under the Hood
-
-- Switch from hologram to mashumaro jsonschema ([#8426](https://github.com/dbt-labs/dbt-core/issues/8426))
-- Refactor flaky test pp_versioned_models ([#7781](https://github.com/dbt-labs/dbt-core/issues/7781))
-- format exception from dbtPlugin.initialize ([#8152](https://github.com/dbt-labs/dbt-core/issues/8152))
-- A way to control maxBytes for a single dbt.log file ([#8199](https://github.com/dbt-labs/dbt-core/issues/8199))
-- Ref expressions with version can now be processed by the latest version of the high-performance dbt-extractor library. ([#7688](https://github.com/dbt-labs/dbt-core/issues/7688))
-- Bump manifest schema version to v11, freeze manifest v10 ([#8333](https://github.com/dbt-labs/dbt-core/issues/8333))
-- add tracking for plugin.get_nodes calls ([#8344](https://github.com/dbt-labs/dbt-core/issues/8344))
-- add internal flag: --no-partial-parse-file-diff to inform whether to compute a file diff during partial parsing ([#8363](https://github.com/dbt-labs/dbt-core/issues/8363))
-- Add return values to a number of functions for mypy ([#8389](https://github.com/dbt-labs/dbt-core/issues/8389))
-- Fix mypy warnings for ManifestLoader.load() ([#8401](https://github.com/dbt-labs/dbt-core/issues/8401))
-- Use python version 3.10.7 in Docker image. ([#8444](https://github.com/dbt-labs/dbt-core/issues/8444))
-- Re-organize jinja macros: relation-specific in /macros/adapters/relations/<relation>, relation agnostic in /macros/relations ([#8449](https://github.com/dbt-labs/dbt-core/issues/8449))
-- Update typing to meet mypy standards ([#8396](https://github.com/dbt-labs/dbt-core/issues/8396))
-- Mypy errors - adapters/factory.py ([#8387](https://github.com/dbt-labs/dbt-core/issues/8387))
-- Added more type annotations. ([#8537](https://github.com/dbt-labs/dbt-core/issues/8537))
-- Audit potential circular dependencies ([#8349](https://github.com/dbt-labs/dbt-core/issues/8349))
-- Add functional test for advanced ref override ([#8566](https://github.com/dbt-labs/dbt-core/issues/8566))
-- Add typing to __init__ in base.py ([#8398](https://github.com/dbt-labs/dbt-core/issues/8398))
-- Fix untyped functions in task/runnable.py (mypy warning) ([#8402](https://github.com/dbt-labs/dbt-core/issues/8402))
-- add a test for ephemeral cte injection ([#8225](https://github.com/dbt-labs/dbt-core/issues/8225))
-- Fix test_numeric_values to look for more specific strings ([#8470](https://github.com/dbt-labs/dbt-core/issues/8470))
-- Pin types-requests<2.31.0 in `dev-requirements.txt` ([#8789](https://github.com/dbt-labs/dbt-core/issues/8789))
-- Add warning_tag to UnversionedBreakingChange ([#8827](https://github.com/dbt-labs/dbt-core/issues/8827))
-- Update v10 manifest schema to match 1.6 for testing schema compatibility ([#8835](https://github.com/dbt-labs/dbt-core/issues/8835))
-- Add a no-op runner for Saved Qeury ([#8893](https://github.com/dbt-labs/dbt-core/issues/8893))
-
-### Dependencies
-
-- Bump mypy from 1.3.0 to 1.4.0 ([#7912](https://github.com/dbt-labs/dbt-core/pull/7912))
-- Bump mypy from 1.4.0 to 1.4.1 ([#8219](https://github.com/dbt-labs/dbt-core/pull/8219))
-- Update pin for click<9 ([#8232](https://github.com/dbt-labs/dbt-core/pull/8232))
-- Add upper bound to sqlparse pin of <0.5 ([#8236](https://github.com/dbt-labs/dbt-core/pull/8236))
-- Support dbt-semantic-interfaces 0.2.0 ([#8250](https://github.com/dbt-labs/dbt-core/pull/8250))
-- Bump docker/build-push-action from 4 to 5 ([#8783](https://github.com/dbt-labs/dbt-core/pull/8783))
-- Upgrade dbt-semantic-interfaces dep to 0.3.0 ([#8819](https://github.com/dbt-labs/dbt-core/pull/8819))
-- Begin using DSI 0.4.x ([#8892](https://github.com/dbt-labs/dbt-core/pull/8892))
-
-### Contributors
-- [@anjutiwari](https://github.com/anjutiwari) ([#7428](https://github.com/dbt-labs/dbt-core/issues/7428), [#8223](https://github.com/dbt-labs/dbt-core/issues/8223))
-- [@benmosher](https://github.com/benmosher) ([#8480](https://github.com/dbt-labs/dbt-core/issues/8480))
-- [@d-kaneshiro](https://github.com/d-kaneshiro) ([#None](https://github.com/dbt-labs/dbt-core/issues/None))
-- [@dave-connors-3](https://github.com/dave-connors-3) ([#8153](https://github.com/dbt-labs/dbt-core/issues/8153), [#8589](https://github.com/dbt-labs/dbt-core/issues/8589))
-- [@dylan-murray](https://github.com/dylan-murray) ([#8683](https://github.com/dbt-labs/dbt-core/issues/8683))
-- [@ezraerb](https://github.com/ezraerb) ([#6154](https://github.com/dbt-labs/dbt-core/issues/6154))
-- [@gem7318](https://github.com/gem7318) ([#8010](https://github.com/dbt-labs/dbt-core/issues/8010))
-- [@jamezrin](https://github.com/jamezrin) ([#8211](https://github.com/dbt-labs/dbt-core/issues/8211))
-- [@jusbaldw](https://github.com/jusbaldw) ([#6643](https://github.com/dbt-labs/dbt-core/issues/6643))
-- [@lllong33](https://github.com/lllong33) ([#5301](https://github.com/dbt-labs/dbt-core/issues/5301))
-- [@marcodamore](https://github.com/marcodamore) ([#436](https://github.com/dbt-labs/dbt-core/issues/436))
-- [@mescanne](https://github.com/mescanne) ([#8614](https://github.com/dbt-labs/dbt-core/issues/8614))
-- [@pgoslatara](https://github.com/pgoslatara) ([#8100](https://github.com/dbt-labs/dbt-core/issues/8100))
-- [@philippeboyd](https://github.com/philippeboyd) ([#5374](https://github.com/dbt-labs/dbt-core/issues/5374))
-- [@ramonvermeulen](https://github.com/ramonvermeulen) ([#3990](https://github.com/dbt-labs/dbt-core/issues/3990))
-- [@renanleme](https://github.com/renanleme) ([#8692](https://github.com/dbt-labs/dbt-core/issues/8692))
-
 ## Previous Releases
 
 For information on prior major and minor releases, see their changelogs:

Dockerfile.test

@@ -33,6 +33,9 @@ RUN apt-get update \
     python-is-python3 \
     python-dev-is-python3 \
     python3-pip \
+    python3.8 \
+    python3.8-dev \
+    python3.8-venv \
     python3.9 \
     python3.9-dev \
     python3.9-venv \

codecov.yml

@@ -1,13 +0,0 @@
-ignore:
-  - ".github"
-  - ".changes"
-coverage:
-  status:
-    project:
-      default:
-        target: auto
-        threshold: 0.1% # Reduce noise by ignoring rounding errors in coverage drops
-    patch:
-      default:
-        target: auto
-        threshold: 80%

core/dbt/adapters/base/column.py

@@ -7,12 +7,12 @@ from dbt.exceptions import DbtRuntimeError
 
 @dataclass
 class Column:
-    # Note: This is automatically used by contract code
-    # No-op conversions (INTEGER => INT) have been removed.
-    # Any adapter that wants to take advantage of "translate_type"
-    # should create a ClassVar with the appropriate conversions.
     TYPE_LABELS: ClassVar[Dict[str, str]] = {
         "STRING": "TEXT",
+        "TIMESTAMP": "TIMESTAMP",
+        "FLOAT": "FLOAT",
+        "INTEGER": "INT",
+        "BOOLEAN": "BOOLEAN",
     }
     column: str
     dtype: str

core/dbt/adapters/base/connections.py

@@ -72,7 +72,7 @@ class BaseConnectionManager(metaclass=abc.ABCMeta):
 
     TYPE: str = NotImplemented
 
-    def __init__(self, profile: AdapterRequiredConfig) -> None:
+    def __init__(self, profile: AdapterRequiredConfig):
         self.profile = profile
         self.thread_connections: Dict[Hashable, Connection] = {}
         self.lock: RLock = flags.MP_CONTEXT.RLock()
@@ -400,7 +400,7 @@ class BaseConnectionManager(metaclass=abc.ABCMeta):
 
     @abc.abstractmethod
     def execute(
-        self, sql: str, auto_begin: bool = False, fetch: bool = False, limit: Optional[int] = None
+        self, sql: str, auto_begin: bool = False, fetch: bool = False
     ) -> Tuple[AdapterResponse, agate.Table]:
         """Execute the given SQL.
 
@@ -408,28 +408,7 @@ class BaseConnectionManager(metaclass=abc.ABCMeta):
         :param bool auto_begin: If set, and dbt is not currently inside a
             transaction, automatically begin one.
         :param bool fetch: If set, fetch results.
-        :param int limit: If set, limits the result set
         :return: A tuple of the query status and results (empty if fetch=False).
         :rtype: Tuple[AdapterResponse, agate.Table]
         """
         raise dbt.exceptions.NotImplementedError("`execute` is not implemented for this adapter!")
-
-    def add_select_query(self, sql: str) -> Tuple[Connection, Any]:
-        """
-        This was added here because base.impl.BaseAdapter.get_column_schema_from_query expects it to be here.
-        That method wouldn't work unless the adapter used sql.impl.SQLAdapter, sql.connections.SQLConnectionManager
-        or defined this method on <Adapter>ConnectionManager before passing it in to <Adapter>Adapter.
-
-        See https://github.com/dbt-labs/dbt-core/issues/8396 for more information.
-        """
-        raise dbt.exceptions.NotImplementedError(
-            "`add_select_query` is not implemented for this adapter!"
-        )
-
-    @classmethod
-    def data_type_code_to_name(cls, type_code: Union[int, str]) -> str:
-        """Get the string representation of the data type from the type_code."""
-        # https://peps.python.org/pep-0249/#type-objects
-        raise dbt.exceptions.NotImplementedError(
-            "`data_type_code_to_name` is not implemented for this adapter!"
-        )

core/dbt/adapters/base/impl.py

@@ -17,11 +17,9 @@ from typing import (
     Set,
     Tuple,
     Type,
-    TypedDict,
     Union,
 )
 
-from dbt.adapters.capability import Capability, CapabilityDict
 from dbt.contracts.graph.nodes import ColumnLevelConstraint, ConstraintType, ModelLevelConstraint
 
 import agate
@@ -45,14 +43,8 @@ from dbt.exceptions import (
     UnexpectedNullError,
 )
 
-from dbt.adapters.protocol import AdapterConfig
+from dbt.adapters.protocol import AdapterConfig, ConnectionManagerProtocol
-from dbt.clients.agate_helper import (
+from dbt.clients.agate_helper import empty_table, merge_tables, table_from_rows
-    empty_table,
-    get_column_value_uncased,
-    merge_tables,
-    table_from_rows,
-    Integer,
-)
 from dbt.clients.jinja import MacroGenerator
 from dbt.contracts.graph.manifest import Manifest, MacroManifest
 from dbt.contracts.graph.nodes import ResultNode
@@ -68,7 +60,7 @@ from dbt.events.types import (
 )
 from dbt.utils import filter_null_values, executor, cast_to_str, AttrDict
 
-from dbt.adapters.base.connections import Connection, AdapterResponse, BaseConnectionManager
+from dbt.adapters.base.connections import Connection, AdapterResponse
 from dbt.adapters.base.meta import AdapterMeta, available
 from dbt.adapters.base.relation import (
     ComponentName,
@@ -82,9 +74,7 @@ from dbt.adapters.cache import RelationsCache, _make_ref_key_dict
 from dbt import deprecations
 
 GET_CATALOG_MACRO_NAME = "get_catalog"
-GET_CATALOG_RELATIONS_MACRO_NAME = "get_catalog_relations"
 FRESHNESS_MACRO_NAME = "collect_freshness"
-GET_RELATION_LAST_MODIFIED_MACRO_NAME = "get_relation_last_modified"
 
 
 class ConstraintSupport(str, Enum):
@@ -119,7 +109,7 @@ def _catalog_filter_schemas(manifest: Manifest) -> Callable[[agate.Row], bool]:
     return test
 
 
-def _utc(dt: Optional[datetime], source: Optional[BaseRelation], field_name: str) -> datetime:
+def _utc(dt: Optional[datetime], source: BaseRelation, field_name: str) -> datetime:
     """If dt has a timezone, return a new datetime that's in UTC. Otherwise,
     assume the datetime is already for UTC and add the timezone.
     """
@@ -171,12 +161,6 @@ class PythonJobHelper:
         raise NotImplementedError("PythonJobHelper submit function is not implemented yet")
 
 
-class FreshnessResponse(TypedDict):
-    max_loaded_at: datetime
-    snapshotted_at: datetime
-    age: float  # age in seconds
-
-
 class BaseAdapter(metaclass=AdapterMeta):
     """The BaseAdapter provides an abstract base class for adapters.
 
@@ -224,7 +208,7 @@ class BaseAdapter(metaclass=AdapterMeta):
 
     Relation: Type[BaseRelation] = BaseRelation
     Column: Type[BaseColumn] = BaseColumn
-    ConnectionManager: Type[BaseConnectionManager]
+    ConnectionManager: Type[ConnectionManagerProtocol]
 
     # A set of clobber config fields accepted by this adapter
     # for use in materializations
@@ -238,11 +222,7 @@ class BaseAdapter(metaclass=AdapterMeta):
         ConstraintType.foreign_key: ConstraintSupport.ENFORCED,
     }
 
-    # This static member variable can be overriden in concrete adapter
+    def __init__(self, config):
-    # implementations to indicate adapter support for optional capabilities.
-    _capabilities = CapabilityDict({})
-
-    def __init__(self, config) -> None:
         self.config = config
         self.cache = RelationsCache()
         self.connections = self.ConnectionManager(config)
@@ -335,21 +315,14 @@ class BaseAdapter(metaclass=AdapterMeta):
 
     @available.parse(lambda *a, **k: ("", empty_table()))
     def get_partitions_metadata(self, table: str) -> Tuple[agate.Table]:
-        """
+        """Obtain partitions metadata for a BigQuery partitioned table.
-        TODO: Can we move this to dbt-bigquery?
-        Obtain partitions metadata for a BigQuery partitioned table.
 
-        :param str table: a partitioned table id, in standard SQL format.
+        :param str table_id: a partitioned table id, in standard SQL format.
         :return: a partition metadata tuple, as described in
             https://cloud.google.com/bigquery/docs/creating-partitioned-tables#getting_partition_metadata_using_meta_tables.
         :rtype: agate.Table
         """
-        if hasattr(self.connections, "get_partitions_metadata"):
+        return self.connections.get_partitions_metadata(table=table)
-            return self.connections.get_partitions_metadata(table=table)
-        else:
-            raise NotImplementedError(
-                "`get_partitions_metadata` is not implemented for this adapter!"
-            )
 
     ###
     # Methods that should never be overridden
@@ -435,30 +408,7 @@ class BaseAdapter(metaclass=AdapterMeta):
         lowercase strings.
         """
         info_schema_name_map = SchemaSearchMap()
-        relations = self._get_catalog_relations(manifest)
+        nodes: Iterator[ResultNode] = chain(
-        for relation in relations:
-            info_schema_name_map.add(relation)
-        # result is a map whose keys are information_schema Relations without
-        # identifiers that have appropriate database prefixes, and whose values
-        # are sets of lowercase schema names that are valid members of those
-        # databases
-        return info_schema_name_map
-
-    def _get_catalog_relations_by_info_schema(
-        self, relations
-    ) -> Dict[InformationSchema, List[BaseRelation]]:
-        relations_by_info_schema: Dict[InformationSchema, List[BaseRelation]] = dict()
-        for relation in relations:
-            info_schema = relation.information_schema_only()
-            if info_schema not in relations_by_info_schema:
-                relations_by_info_schema[info_schema] = []
-            relations_by_info_schema[info_schema].append(relation)
-
-        return relations_by_info_schema
-
-    def _get_catalog_relations(self, manifest: Manifest) -> List[BaseRelation]:
-
-        nodes = chain(
             [
                 node
                 for node in manifest.nodes.values()
@@ -466,9 +416,14 @@ class BaseAdapter(metaclass=AdapterMeta):
             ],
             manifest.sources.values(),
         )
-
+        for node in nodes:
-        relations = [self.Relation.create_from(self.config, n) for n in nodes]
+            relation = self.Relation.create_from(self.config, node)
-        return relations
+            info_schema_name_map.add(relation)
+        # result is a map whose keys are information_schema Relations without
+        # identifiers that have appropriate database prefixes, and whose values
+        # are sets of lowercase schema names that are valid members of those
+        # databases
+        return info_schema_name_map
 
     def _relations_cache_for_schemas(
         self, manifest: Manifest, cache_schemas: Optional[Set[BaseRelation]] = None
@@ -498,10 +453,9 @@ class BaseAdapter(metaclass=AdapterMeta):
         # it's possible that there were no relations in some schemas. We want
         # to insert the schemas we query into the cache's `.schemas` attribute
         # so we can check it later
-        cache_update: Set[Tuple[Optional[str], str]] = set()
+        cache_update: Set[Tuple[Optional[str], Optional[str]]] = set()
         for relation in cache_schemas:
-            if relation.schema:
+            cache_update.add((relation.database, relation.schema))
-                cache_update.add((relation.database, relation.schema))
         self.cache.update_schemas(cache_update)
 
     def set_relations_cache(
@@ -963,17 +917,6 @@ class BaseAdapter(metaclass=AdapterMeta):
         """
         raise NotImplementedError("`convert_number_type` is not implemented for this adapter!")
 
-    @classmethod
-    def convert_integer_type(cls, agate_table: agate.Table, col_idx: int) -> str:
-        """Return the type in the database that best maps to the agate.Number
-        type for the given agate table and column index.
-
-        :param agate_table: The table
-        :param col_idx: The index into the agate table for the column.
-        :return: The name of the type in the database
-        """
-        return "integer"
-
     @classmethod
     @abc.abstractmethod
     def convert_boolean_type(cls, agate_table: agate.Table, col_idx: int) -> str:
@@ -1031,7 +974,6 @@ class BaseAdapter(metaclass=AdapterMeta):
     def convert_agate_type(cls, agate_table: agate.Table, col_idx: int) -> Optional[str]:
         agate_type: Type = agate_table.column_types[col_idx]
         conversions: List[Tuple[Type, Callable[..., str]]] = [
-            (Integer, cls.convert_integer_type),
             (agate.Text, cls.convert_text_type),
             (agate.Number, cls.convert_number_type),
             (agate.Boolean, cls.convert_boolean_type),
@@ -1143,107 +1085,24 @@ class BaseAdapter(metaclass=AdapterMeta):
         results = self._catalog_filter_table(table, manifest)  # type: ignore[arg-type]
         return results
 
-    def _get_one_catalog_by_relations(
-        self,
-        information_schema: InformationSchema,
-        relations: List[BaseRelation],
-        manifest: Manifest,
-    ) -> agate.Table:
-
-        kwargs = {
-            "information_schema": information_schema,
-            "relations": relations,
-        }
-        table = self.execute_macro(
-            GET_CATALOG_RELATIONS_MACRO_NAME,
-            kwargs=kwargs,
-            # pass in the full manifest, so we get any local project
-            # overrides
-            manifest=manifest,
-        )
-
-        results = self._catalog_filter_table(table, manifest)  # type: ignore[arg-type]
-        return results
-
-    def get_filtered_catalog(
-        self, manifest: Manifest, relations: Optional[Set[BaseRelation]] = None
-    ):
-        catalogs: agate.Table
-        if (
-            relations is None
-            or len(relations) > 100
-            or not self.supports(Capability.SchemaMetadataByRelations)
-        ):
-            # Do it the traditional way. We get the full catalog.
-            catalogs, exceptions = self.get_catalog(manifest)
-        else:
-            # Do it the new way. We try to save time by selecting information
-            # only for the exact set of relations we are interested in.
-            catalogs, exceptions = self.get_catalog_by_relations(manifest, relations)
-
-        if relations and catalogs:
-            relation_map = {
-                (
-                    r.database.casefold() if r.database else None,
-                    r.schema.casefold() if r.schema else None,
-                    r.identifier.casefold() if r.identifier else None,
-                )
-                for r in relations
-            }
-
-            def in_map(row: agate.Row):
-                d = _expect_row_value("table_database", row)
-                s = _expect_row_value("table_schema", row)
-                i = _expect_row_value("table_name", row)
-                d = d.casefold() if d is not None else None
-                s = s.casefold() if s is not None else None
-                i = i.casefold() if i is not None else None
-                return (d, s, i) in relation_map
-
-            catalogs = catalogs.where(in_map)
-
-        return catalogs, exceptions
-
-    def row_matches_relation(self, row: agate.Row, relations: Set[BaseRelation]):
-        pass
-
     def get_catalog(self, manifest: Manifest) -> Tuple[agate.Table, List[Exception]]:
+        schema_map = self._get_catalog_schemas(manifest)
+
         with executor(self.config) as tpe:
             futures: List[Future[agate.Table]] = []
-            schema_map: SchemaSearchMap = self._get_catalog_schemas(manifest)
             for info, schemas in schema_map.items():
                 if len(schemas) == 0:
                     continue
                 name = ".".join([str(info.database), "information_schema"])
+
                 fut = tpe.submit_connected(
                     self, name, self._get_one_catalog, info, schemas, manifest
                 )
                 futures.append(fut)
 
-        catalogs, exceptions = catch_as_completed(futures)
-        return catalogs, exceptions
-
-    def get_catalog_by_relations(
-        self, manifest: Manifest, relations: Set[BaseRelation]
-    ) -> Tuple[agate.Table, List[Exception]]:
-        with executor(self.config) as tpe:
-            futures: List[Future[agate.Table]] = []
-            relations_by_schema = self._get_catalog_relations_by_info_schema(relations)
-            for info_schema in relations_by_schema:
-                name = ".".join([str(info_schema.database), "information_schema"])
-                relations = set(relations_by_schema[info_schema])
-                fut = tpe.submit_connected(
-                    self,
-                    name,
-                    self._get_one_catalog_by_relations,
-                    info_schema,
-                    relations,
-                    manifest,
-                )
-                futures.append(fut)
-
             catalogs, exceptions = catch_as_completed(futures)
-            return catalogs, exceptions
+
+        return catalogs, exceptions
 
     def cancel_open_connections(self):
         """Cancel all open connections."""
@@ -1255,7 +1114,7 @@ class BaseAdapter(metaclass=AdapterMeta):
         loaded_at_field: str,
         filter: Optional[str],
         manifest: Optional[Manifest] = None,
-    ) -> Tuple[Optional[AdapterResponse], FreshnessResponse]:
+    ) -> Tuple[Optional[AdapterResponse], Dict[str, Any]]:
         """Calculate the freshness of sources in dbt, and return it"""
         kwargs: Dict[str, Any] = {
             "source": source,
@@ -1290,52 +1149,13 @@ class BaseAdapter(metaclass=AdapterMeta):
 
         snapshotted_at = _utc(table[0][1], source, loaded_at_field)
         age = (snapshotted_at - max_loaded_at).total_seconds()
-        freshness: FreshnessResponse = {
+        freshness = {
             "max_loaded_at": max_loaded_at,
             "snapshotted_at": snapshotted_at,
             "age": age,
         }
         return adapter_response, freshness
 
-    def calculate_freshness_from_metadata(
-        self,
-        source: BaseRelation,
-        manifest: Optional[Manifest] = None,
-    ) -> Tuple[Optional[AdapterResponse], FreshnessResponse]:
-        kwargs: Dict[str, Any] = {
-            "information_schema": source.information_schema_only(),
-            "relations": [source],
-        }
-        result = self.execute_macro(
-            GET_RELATION_LAST_MODIFIED_MACRO_NAME, kwargs=kwargs, manifest=manifest
-        )
-        adapter_response, table = result.response, result.table  # type: ignore[attr-defined]
-
-        try:
-            row = table[0]
-            last_modified_val = get_column_value_uncased("last_modified", row)
-            snapshotted_at_val = get_column_value_uncased("snapshotted_at", row)
-        except Exception:
-            raise MacroResultError(GET_RELATION_LAST_MODIFIED_MACRO_NAME, table)
-
-        if last_modified_val is None:
-            # Interpret missing value as "infinitely long ago"
-            max_loaded_at = datetime(1, 1, 1, 0, 0, 0, tzinfo=pytz.UTC)
-        else:
-            max_loaded_at = _utc(last_modified_val, None, "last_modified")
-
-        snapshotted_at = _utc(snapshotted_at_val, None, "snapshotted_at")
-
-        age = (snapshotted_at - max_loaded_at).total_seconds()
-
-        freshness: FreshnessResponse = {
-            "max_loaded_at": max_loaded_at,
-            "snapshotted_at": snapshotted_at,
-            "age": age,
-        }
-
-        return adapter_response, freshness
-
     def pre_model_hook(self, config: Mapping[str, Any]) -> Any:
         """A hook for running some operation before the model materialization
         runs. The hook can assume it has a connection available.
@@ -1609,14 +1429,6 @@ class BaseAdapter(metaclass=AdapterMeta):
         else:
             return None
 
-    @classmethod
-    def capabilities(cls) -> CapabilityDict:
-        return cls._capabilities
-
-    @classmethod
-    def supports(cls, capability: Capability) -> bool:
-        return bool(cls.capabilities()[capability])
-
 
 COLUMNS_EQUAL_SQL = """
 with diff_count as (

core/dbt/adapters/base/meta.py

@@ -93,7 +93,7 @@ class AdapterMeta(abc.ABCMeta):
     _available_: FrozenSet[str]
     _parse_replacements_: Dict[str, Callable]
 
-    def __new__(mcls, name, bases, namespace, **kwargs) -> "AdapterMeta":
+    def __new__(mcls, name, bases, namespace, **kwargs):
         # mypy does not like the `**kwargs`. But `ABCMeta` itself takes
         # `**kwargs` in its argspec here (and passes them to `type.__new__`.
         # I'm not sure there is any benefit to it after poking around a bit,

core/dbt/adapters/base/plugin.py

@@ -29,7 +29,7 @@ class AdapterPlugin:
         credentials: Type[Credentials],
         include_path: str,
         dependencies: Optional[List[str]] = None,
-    ) -> None:
+    ):
 
         self.adapter: Type[AdapterProtocol] = adapter
         self.credentials: Type[Credentials] = credentials

core/dbt/adapters/base/query_headers.py

@@ -11,7 +11,7 @@ from dbt.exceptions import DbtRuntimeError
 
 
 class NodeWrapper:
-    def __init__(self, node) -> None:
+    def __init__(self, node):
         self._inner_node = node
 
     def __getattr__(self, name):
@@ -25,9 +25,9 @@ class _QueryComment(local):
         - a source_name indicating what set the current thread's query comment
     """
 
-    def __init__(self, initial) -> None:
+    def __init__(self, initial):
         self.query_comment: Optional[str] = initial
-        self.append: bool = False
+        self.append = False
 
     def add(self, sql: str) -> str:
         if not self.query_comment:
@@ -57,7 +57,7 @@ QueryStringFunc = Callable[[str, Optional[NodeWrapper]], str]
 
 
 class MacroQueryStringSetter:
-    def __init__(self, config: AdapterRequiredConfig, manifest: Manifest) -> None:
+    def __init__(self, config: AdapterRequiredConfig, manifest: Manifest):
         self.manifest = manifest
         self.config = config
 

core/dbt/adapters/base/relation.py

@@ -1,6 +1,6 @@
 from collections.abc import Hashable
 from dataclasses import dataclass, field
-from typing import Optional, TypeVar, Any, Type, Dict, Iterator, Tuple, Set, Union, FrozenSet
+from typing import Optional, TypeVar, Any, Type, Dict, Iterator, Tuple, Set
 
 from dbt.contracts.graph.nodes import SourceDefinition, ManifestNode, ResultNode, ParsedNode
 from dbt.contracts.relation import (
@@ -23,7 +23,6 @@ import dbt.exceptions
 
 
 Self = TypeVar("Self", bound="BaseRelation")
-SerializableIterable = Union[Tuple, FrozenSet]
 
 
 @dataclass(frozen=True, eq=False, repr=False)
@@ -37,18 +36,6 @@ class BaseRelation(FakeAPIObject, Hashable):
     quote_policy: Policy = field(default_factory=lambda: Policy())
     dbt_created: bool = False
 
-    # register relation types that can be renamed for the purpose of replacing relations using stages and backups
-    # adding a relation type here also requires defining the associated rename macro
-    # e.g. adding RelationType.View in dbt-postgres requires that you define:
-    # include/postgres/macros/relations/view/rename.sql::postgres__get_rename_view_sql()
-    renameable_relations: SerializableIterable = field(default_factory=frozenset)
-
-    # register relation types that are atomically replaceable, e.g. they have "create or replace" syntax
-    # adding a relation type here also requires defining the associated replace macro
-    # e.g. adding RelationType.View in dbt-postgres requires that you define:
-    # include/postgres/macros/relations/view/replace.sql::postgres__get_replace_view_sql()
-    replaceable_relations: SerializableIterable = field(default_factory=frozenset)
-
     def _is_exactish_match(self, field: ComponentName, value: str) -> bool:
         if self.dbt_created and self.quote_policy.get_part(field) is False:
             return self.path.get_lowered_part(field) == value.lower()
@@ -182,6 +169,7 @@ class BaseRelation(FakeAPIObject, Hashable):
         return self.include(identifier=False).replace_path(identifier=None)
 
     def _render_iterator(self) -> Iterator[Tuple[Optional[ComponentName], Optional[str]]]:
+
         for key in ComponentName:
             path_part: Optional[str] = None
             if self.include_policy.get_part(key):
@@ -298,14 +286,6 @@ class BaseRelation(FakeAPIObject, Hashable):
         )
         return cls.from_dict(kwargs)
 
-    @property
-    def can_be_renamed(self) -> bool:
-        return self.type in self.renameable_relations
-
-    @property
-    def can_be_replaced(self) -> bool:
-        return self.type in self.replaceable_relations
-
     def __repr__(self) -> str:
         return "<{} {}>".format(self.__class__.__name__, self.render())
 
@@ -459,11 +439,11 @@ class SchemaSearchMap(Dict[InformationSchema, Set[Optional[str]]]):
         self[key].add(schema)
 
     def search(self) -> Iterator[Tuple[InformationSchema, Optional[str]]]:
-        for information_schema, schemas in self.items():
+        for information_schema_name, schemas in self.items():
             for schema in schemas:
-                yield information_schema, schema
+                yield information_schema_name, schema
 
-    def flatten(self, allow_multiple_databases: bool = False) -> "SchemaSearchMap":
+    def flatten(self, allow_multiple_databases: bool = False):
         new = self.__class__()
 
         # make sure we don't have multiple databases if allow_multiple_databases is set to False

core/dbt/adapters/cache.py

@@ -38,8 +38,8 @@ class _CachedRelation:
     :attr BaseRelation inner: The underlying dbt relation.
     """
 
-    def __init__(self, inner) -> None:
+    def __init__(self, inner):
-        self.referenced_by: Dict[_ReferenceKey, _CachedRelation] = {}
+        self.referenced_by = {}
         self.inner = inner
 
     def __str__(self) -> str:

core/dbt/adapters/capability.py

@@ -1,52 +0,0 @@
-from dataclasses import dataclass
-from enum import Enum
-from typing import Optional, DefaultDict, Mapping
-
-
-class Capability(str, Enum):
-    """Enumeration of optional adapter features which can be probed using BaseAdapter.has_feature()"""
-
-    SchemaMetadataByRelations = "SchemaMetadataByRelations"
-    """Indicates efficient support for retrieving schema metadata for a list of relations, rather than always retrieving
-    all the relations in a schema."""
-
-    TableLastModifiedMetadata = "TableLastModifiedMetadata"
-    """Indicates support for determining the time of the last table modification by querying database metadata."""
-
-
-class Support(str, Enum):
-    Unknown = "Unknown"
-    """The adapter has not declared whether this capability is a feature of the underlying DBMS."""
-
-    Unsupported = "Unsupported"
-    """This capability is not possible with the underlying DBMS, so the adapter does not implement related macros."""
-
-    NotImplemented = "NotImplemented"
-    """This capability is available in the underlying DBMS, but support has not yet been implemented in the adapter."""
-
-    Versioned = "Versioned"
-    """Some versions of the DBMS supported by the adapter support this capability and the adapter has implemented any
-    macros needed to use it."""
-
-    Full = "Full"
-    """All versions of the DBMS supported by the adapter support this capability and the adapter has implemented any
-    macros needed to use it."""
-
-
-@dataclass
-class CapabilitySupport:
-    support: Support
-    first_version: Optional[str] = None
-
-    def __bool__(self):
-        return self.support == Support.Versioned or self.support == Support.Full
-
-
-class CapabilityDict(DefaultDict[Capability, CapabilitySupport]):
-    def __init__(self, vals: Mapping[Capability, CapabilitySupport]):
-        super().__init__(self._default)
-        self.update(vals)
-
-    @staticmethod
-    def _default():
-        return CapabilitySupport(support=Support.Unknown)

core/dbt/adapters/factory.py

@@ -19,7 +19,7 @@ Adapter = AdapterProtocol
 
 
 class AdapterContainer:
-    def __init__(self) -> None:
+    def __init__(self):
         self.lock = threading.Lock()
         self.adapters: Dict[str, Adapter] = {}
         self.plugins: Dict[str, AdapterPlugin] = {}

core/dbt/adapters/protocol.py

@@ -90,7 +90,7 @@ class AdapterProtocol(  # type: ignore[misc]
     ConnectionManager: Type[ConnectionManager_T]
     connections: ConnectionManager_T
 
-    def __init__(self, config: AdapterRequiredConfig) -> None:
+    def __init__(self, config: AdapterRequiredConfig):
         ...
 
     @classmethod

core/dbt/adapters/sql/connections.py

@@ -1,6 +1,6 @@
 import abc
 import time
-from typing import List, Optional, Tuple, Any, Iterable, Dict
+from typing import List, Optional, Tuple, Any, Iterable, Dict, Union
 
 import agate
 
@@ -131,6 +131,14 @@ class SQLConnectionManager(BaseConnectionManager):
 
         return dbt.clients.agate_helper.table_from_data_flat(data, column_names)
 
+    @classmethod
+    def data_type_code_to_name(cls, type_code: Union[int, str]) -> str:
+        """Get the string representation of the data type from the type_code."""
+        # https://peps.python.org/pep-0249/#type-objects
+        raise dbt.exceptions.NotImplementedError(
+            "`data_type_code_to_name` is not implemented for this adapter!"
+        )
+
     def execute(
         self, sql: str, auto_begin: bool = False, fetch: bool = False, limit: Optional[int] = None
     ) -> Tuple[AdapterResponse, agate.Table]:

core/dbt/adapters/sql/impl.py

@@ -75,10 +75,6 @@ class SQLAdapter(BaseAdapter):
         decimals = agate_table.aggregate(agate.MaxPrecision(col_idx))  # type: ignore[attr-defined]
         return "float8" if decimals else "integer"
 
-    @classmethod
-    def convert_integer_type(cls, agate_table: agate.Table, col_idx: int) -> str:
-        return "integer"
-
     @classmethod
     def convert_boolean_type(cls, agate_table: agate.Table, col_idx: int) -> str:
         return "boolean"

core/dbt/cli/flags.py

@@ -3,7 +3,6 @@ import sys
 from dataclasses import dataclass
 from importlib import import_module
 from multiprocessing import get_context
-from pathlib import Path
 from pprint import pformat as pf
 from typing import Any, Callable, Dict, List, Optional, Set, Union
 
@@ -12,10 +11,10 @@ from click.core import Command as ClickCommand, Group, ParameterSource
 from dbt.cli.exceptions import DbtUsageException
 from dbt.cli.resolvers import default_log_path, default_project_dir
 from dbt.cli.types import Command as CliCommand
-from dbt.config.project import read_project_flags
+from dbt.config.profile import read_user_config
-from dbt.contracts.project import ProjectFlags
+from dbt.contracts.project import UserConfig
 from dbt.exceptions import DbtInternalError
-from dbt.deprecations import fire_buffered_deprecations, renamed_env_var
+from dbt.deprecations import renamed_env_var
 from dbt.helper_types import WarnErrorOptions
 
 if os.name != "nt":
@@ -25,8 +24,7 @@ if os.name != "nt":
 FLAGS_DEFAULTS = {
     "INDIRECT_SELECTION": "eager",
     "TARGET_PATH": None,
-    "WARN_ERROR": None,
+    # Cli args without user_config or env var option.
-    # Cli args without project_flags or env var option.
     "FULL_REFRESH": False,
     "STRICT_MODE": False,
     "STORE_FAILURES": False,
@@ -59,10 +57,11 @@ def args_to_context(args: List[str]) -> Context:
     from dbt.cli.main import cli
 
     cli_ctx = cli.make_context(cli.name, args)
-    # Split args if they're a comma separated string.
+    # Split args if they're a comma seperated string.
     if len(args) == 1 and "," in args[0]:
         args = args[0].split(",")
     sub_command_name, sub_command, args = cli.resolve_command(cli_ctx, args)
+
     # Handle source and docs group.
     if isinstance(sub_command, Group):
         sub_command_name, sub_command, args = sub_command.resolve_command(cli_ctx, args)
@@ -78,8 +77,9 @@ class Flags:
     """Primary configuration artifact for running dbt"""
 
     def __init__(
-        self, ctx: Optional[Context] = None, project_flags: Optional[ProjectFlags] = None
+        self, ctx: Optional[Context] = None, user_config: Optional[UserConfig] = None
     ) -> None:
+
         # Set the default flags.
         for key, value in FLAGS_DEFAULTS.items():
             object.__setattr__(self, key, value)
@@ -121,6 +121,7 @@ class Flags:
                 # respected over DBT_PRINT or --print.
                 new_name: Union[str, None] = None
                 if param_name in DEPRECATED_PARAMS:
+
                     # Deprecated env vars can only be set via env var.
                     # We use the deprecated option in click to serialize the value
                     # from the env var string.
@@ -201,40 +202,27 @@ class Flags:
                 invoked_subcommand_ctx, params_assigned_from_default, deprecated_env_vars
             )
 
-        if not project_flags:
+        if not user_config:
-            project_dir = getattr(self, "PROJECT_DIR", str(default_project_dir()))
             profiles_dir = getattr(self, "PROFILES_DIR", None)
-            if profiles_dir and project_dir:
+            user_config = read_user_config(profiles_dir) if profiles_dir else None
-                project_flags = read_project_flags(project_dir, profiles_dir)
-            else:
-                project_flags = None
 
         # Add entire invocation command to flags
         object.__setattr__(self, "INVOCATION_COMMAND", "dbt " + " ".join(sys.argv[1:]))
 
-        if project_flags:
+        # Overwrite default assignments with user config if available.
-            # Overwrite default assignments with project flags if available.
+        if user_config:
             param_assigned_from_default_copy = params_assigned_from_default.copy()
             for param_assigned_from_default in params_assigned_from_default:
-                project_flags_param_value = getattr(
+                user_config_param_value = getattr(user_config, param_assigned_from_default, None)
-                    project_flags, param_assigned_from_default, None
+                if user_config_param_value is not None:
-                )
-                if project_flags_param_value is not None:
                     object.__setattr__(
                         self,
                         param_assigned_from_default.upper(),
-                        convert_config(param_assigned_from_default, project_flags_param_value),
+                        convert_config(param_assigned_from_default, user_config_param_value),
                     )
                     param_assigned_from_default_copy.remove(param_assigned_from_default)
             params_assigned_from_default = param_assigned_from_default_copy
 
-            # Add project-level flags that are not available as CLI options / env vars
-            for (
-                project_level_flag_name,
-                project_level_flag_value,
-            ) in project_flags.project_only_flags.items():
-                object.__setattr__(self, project_level_flag_name.upper(), project_level_flag_value)
-
         # Set hard coded flags.
         object.__setattr__(self, "WHICH", invoked_subcommand_name or ctx.info_name)
         object.__setattr__(self, "MP_CONTEXT", get_context("spawn"))
@@ -248,11 +236,9 @@ class Flags:
         # Starting in v1.5, if `log-path` is set in `dbt_project.yml`, it will raise a deprecation warning,
         # with the possibility of removing it in a future release.
         if getattr(self, "LOG_PATH", None) is None:
-            project_dir = getattr(self, "PROJECT_DIR", str(default_project_dir()))
+            project_dir = getattr(self, "PROJECT_DIR", default_project_dir())
             version_check = getattr(self, "VERSION_CHECK", True)
-            object.__setattr__(
+            object.__setattr__(self, "LOG_PATH", default_log_path(project_dir, version_check))
-                self, "LOG_PATH", default_log_path(Path(project_dir), version_check)
-            )
 
         # Support console DO NOT TRACK initiative.
         if os.getenv("DO_NOT_TRACK", "").lower() in ("1", "t", "true", "y", "yes"):
@@ -302,8 +288,6 @@ class Flags:
         # not get pickled when written to disk as json.
         object.__delattr__(self, "deprecated_env_var_warnings")
 
-        fire_buffered_deprecations()
-
     @classmethod
     def from_dict(cls, command: CliCommand, args_dict: Dict[str, Any]) -> "Flags":
         command_arg_list = command_params(command, args_dict)
@@ -332,8 +316,10 @@ def command_params(command: CliCommand, args_dict: Dict[str, Any]) -> CommandPar
     default_args = set([x.lower() for x in FLAGS_DEFAULTS.keys()])
 
     res = command.to_list()
+
     for k, v in args_dict.items():
         k = k.lower()
+
         # if a "which" value exists in the args dict, it should match the command provided
         if k == WHICH_KEY:
             if v != command.value:
@@ -343,9 +329,7 @@ def command_params(command: CliCommand, args_dict: Dict[str, Any]) -> CommandPar
             continue
 
         # param was assigned from defaults and should not be included
-        if k not in (cmd_args | prnt_args) or (
+        if k not in (cmd_args | prnt_args) - default_args:
-            k in default_args and v == FLAGS_DEFAULTS[k.upper()]
-        ):
             continue
 
         # if the param is in parent args, it should come before the arg name
@@ -358,14 +342,9 @@ def command_params(command: CliCommand, args_dict: Dict[str, Any]) -> CommandPar
 
         spinal_cased = k.replace("_", "-")
 
-        # MultiOption flags come back as lists, but we want to pass them as space separated strings
-        if isinstance(v, list):
-            v = " ".join(v)
-
         if k == "macro" and command == CliCommand.RUN_OPERATION:
             add_fn(v)
-        # None is a Singleton, False is a Flyweight, only one instance of each.
+        elif v in (None, False):
-        elif v is None or v is False:
             add_fn(f"--no-{spinal_cased}")
         elif v is True:
             add_fn(f"--{spinal_cased}")

core/dbt/cli/main.py

@@ -1,4 +1,3 @@
-import functools
 from copy import copy
 from dataclasses import dataclass
 from typing import Callable, List, Optional, Union
@@ -65,7 +64,7 @@ class dbtRunner:
         self,
         manifest: Optional[Manifest] = None,
         callbacks: Optional[List[Callable[[EventMsg], None]]] = None,
-    ) -> None:
+    ):
         self.manifest = manifest
 
         if callbacks is None:
@@ -119,44 +118,6 @@ class dbtRunner:
             )
 
 
-# approach from https://github.com/pallets/click/issues/108#issuecomment-280489786
-def global_flags(func):
-    @p.cache_selected_only
-    @p.debug
-    @p.deprecated_print
-    @p.enable_legacy_logger
-    @p.fail_fast
-    @p.log_cache_events
-    @p.log_file_max_bytes
-    @p.log_format_file
-    @p.log_level
-    @p.log_level_file
-    @p.log_path
-    @p.macro_debugging
-    @p.partial_parse
-    @p.partial_parse_file_path
-    @p.partial_parse_file_diff
-    @p.populate_cache
-    @p.print
-    @p.printer_width
-    @p.quiet
-    @p.record_timing_info
-    @p.send_anonymous_usage_stats
-    @p.single_threaded
-    @p.static_parser
-    @p.use_colors
-    @p.use_colors_file
-    @p.use_experimental_parser
-    @p.version
-    @p.version_check
-    @p.write_json
-    @functools.wraps(func)
-    def wrapper(*args, **kwargs):
-        return func(*args, **kwargs)
-
-    return wrapper
-
-
 # dbt
 @click.group(
     context_settings={"help_option_names": ["-h", "--help"]},
@@ -165,11 +126,37 @@ def global_flags(func):
     epilog="Specify one of these sub-commands and you can find more help from there.",
 )
 @click.pass_context
-@global_flags
+@p.cache_selected_only
+@p.debug
+@p.deprecated_print
+@p.enable_legacy_logger
+@p.fail_fast
+@p.log_cache_events
+@p.log_file_max_bytes
+@p.log_format
+@p.log_format_file
+@p.log_level
+@p.log_level_file
+@p.log_path
+@p.macro_debugging
+@p.partial_parse
+@p.partial_parse_file_path
+@p.populate_cache
+@p.print
+@p.printer_width
+@p.quiet
+@p.record_timing_info
+@p.send_anonymous_usage_stats
+@p.single_threaded
+@p.static_parser
+@p.use_colors
+@p.use_colors_file
+@p.use_experimental_parser
+@p.version
+@p.version_check
 @p.warn_error
 @p.warn_error_options
-@p.log_format
+@p.write_json
-@p.show_resource_report
 def cli(ctx, **kwargs):
     """An ELT tool for managing your SQL transformations and data models.
     For more documentation on these commands, visit: docs.getdbt.com
@@ -179,14 +166,13 @@ def cli(ctx, **kwargs):
 # dbt build
 @cli.command("build")
 @click.pass_context
-@global_flags
 @p.defer
 @p.deprecated_defer
 @p.exclude
+@p.fail_fast
 @p.favor_state
 @p.deprecated_favor_state
 @p.full_refresh
-@p.include_saved_query
 @p.indirect_selection
 @p.profile
 @p.profiles_dir
@@ -203,6 +189,7 @@ def cli(ctx, **kwargs):
 @p.target_path
 @p.threads
 @p.vars
+@p.version_check
 @requires.postflight
 @requires.preflight
 @requires.profile
@@ -225,8 +212,6 @@ def build(ctx, **kwargs):
 # dbt clean
 @cli.command("clean")
 @click.pass_context
-@global_flags
-@p.clean_project_files_only
 @p.profile
 @p.profiles_dir
 @p.project_dir
@@ -249,7 +234,6 @@ def clean(ctx, **kwargs):
 # dbt docs
 @cli.group()
 @click.pass_context
-@global_flags
 def docs(ctx, **kwargs):
     """Generate or serve the documentation website for your project"""
 
@@ -257,7 +241,6 @@ def docs(ctx, **kwargs):
 # dbt docs generate
 @docs.command("generate")
 @click.pass_context
-@global_flags
 @p.compile_docs
 @p.defer
 @p.deprecated_defer
@@ -270,7 +253,6 @@ def docs(ctx, **kwargs):
 @p.select
 @p.selector
 @p.empty_catalog
-@p.static
 @p.state
 @p.defer_state
 @p.deprecated_state
@@ -278,6 +260,7 @@ def docs(ctx, **kwargs):
 @p.target_path
 @p.threads
 @p.vars
+@p.version_check
 @requires.postflight
 @requires.preflight
 @requires.profile
@@ -300,9 +283,7 @@ def docs_generate(ctx, **kwargs):
 # dbt docs serve
 @docs.command("serve")
 @click.pass_context
-@global_flags
 @p.browser
-@p.host
 @p.port
 @p.profile
 @p.profiles_dir
@@ -330,7 +311,6 @@ def docs_serve(ctx, **kwargs):
 # dbt compile
 @cli.command("compile")
 @click.pass_context
-@global_flags
 @p.defer
 @p.deprecated_defer
 @p.exclude
@@ -349,11 +329,11 @@ def docs_serve(ctx, **kwargs):
 @p.state
 @p.defer_state
 @p.deprecated_state
-@p.compile_inject_ephemeral_ctes
 @p.target
 @p.target_path
 @p.threads
 @p.vars
+@p.version_check
 @requires.postflight
 @requires.preflight
 @requires.profile
@@ -377,7 +357,6 @@ def compile(ctx, **kwargs):
 # dbt show
 @cli.command("show")
 @click.pass_context
-@global_flags
 @p.defer
 @p.deprecated_defer
 @p.exclude
@@ -401,6 +380,7 @@ def compile(ctx, **kwargs):
 @p.target_path
 @p.threads
 @p.vars
+@p.version_check
 @requires.postflight
 @requires.preflight
 @requires.profile
@@ -424,7 +404,6 @@ def show(ctx, **kwargs):
 # dbt debug
 @cli.command("debug")
 @click.pass_context
-@global_flags
 @p.debug_connection
 @p.config_dir
 @p.profile
@@ -432,6 +411,7 @@ def show(ctx, **kwargs):
 @p.project_dir
 @p.target
 @p.vars
+@p.version_check
 @requires.postflight
 @requires.preflight
 def debug(ctx, **kwargs):
@@ -450,46 +430,18 @@ def debug(ctx, **kwargs):
 # dbt deps
 @cli.command("deps")
 @click.pass_context
-@global_flags
 @p.profile
 @p.profiles_dir_exists_false
 @p.project_dir
 @p.target
 @p.vars
-@p.source
-@p.dry_run
-@p.lock
-@p.upgrade
-@p.add_package
 @requires.postflight
 @requires.preflight
 @requires.unset_profile
 @requires.project
 def deps(ctx, **kwargs):
-    """Install dbt packages specified.
+    """Pull the most recent version of the dependencies listed in packages.yml"""
-    In the following case, a new `package-lock.yml` will be generated and the packages are installed:
+    task = DepsTask(ctx.obj["flags"], ctx.obj["project"])
-    - user updated the packages.yml
-    - user specify the flag --update, which means for packages that are specified as a
-      range, dbt-core will try to install the newer version
-    Otherwise, deps will use `package-lock.yml` as source of truth to install packages.
-
-    There is a way to add new packages by providing an `--add-package` flag to deps command
-    which will allow user to specify a package they want to add in the format of packagename@version.
-    """
-    flags = ctx.obj["flags"]
-    if flags.ADD_PACKAGE:
-        if not flags.ADD_PACKAGE["version"] and flags.SOURCE != "local":
-            raise BadOptionUsage(
-                message=f"Version is required in --add-package when a package when source is {flags.SOURCE}",
-                option_name="--add-package",
-            )
-    else:
-        if flags.DRY_RUN:
-            raise BadOptionUsage(
-                message="Invalid flag `--dry-run` when not using `--add-package`.",
-                option_name="--dry-run",
-            )
-    task = DepsTask(flags, ctx.obj["project"])
     results = task.run()
     success = task.interpret_results(results)
     return results, success
@@ -498,7 +450,6 @@ def deps(ctx, **kwargs):
 # dbt init
 @cli.command("init")
 @click.pass_context
-@global_flags
 # for backwards compatibility, accept 'project_name' as an optional positional argument
 @click.argument("project_name", required=False)
 @p.profile
@@ -521,7 +472,6 @@ def init(ctx, **kwargs):
 # dbt list
 @cli.command("list")
 @click.pass_context
-@global_flags
 @p.exclude
 @p.indirect_selection
 @p.models
@@ -567,7 +517,6 @@ cli.add_command(ls, "ls")
 # dbt parse
 @cli.command("parse")
 @click.pass_context
-@global_flags
 @p.profile
 @p.profiles_dir
 @p.project_dir
@@ -575,6 +524,7 @@ cli.add_command(ls, "ls")
 @p.target_path
 @p.threads
 @p.vars
+@p.version_check
 @requires.postflight
 @requires.preflight
 @requires.profile
@@ -584,18 +534,19 @@ cli.add_command(ls, "ls")
 def parse(ctx, **kwargs):
     """Parses the project and provides information on performance"""
     # manifest generation and writing happens in @requires.manifest
+
     return ctx.obj["manifest"], True
 
 
 # dbt run
 @cli.command("run")
 @click.pass_context
-@global_flags
 @p.defer
 @p.deprecated_defer
 @p.favor_state
 @p.deprecated_favor_state
 @p.exclude
+@p.fail_fast
 @p.full_refresh
 @p.profile
 @p.profiles_dir
@@ -609,6 +560,7 @@ def parse(ctx, **kwargs):
 @p.target_path
 @p.threads
 @p.vars
+@p.version_check
 @requires.postflight
 @requires.preflight
 @requires.profile
@@ -631,27 +583,26 @@ def run(ctx, **kwargs):
 # dbt retry
 @cli.command("retry")
 @click.pass_context
-@global_flags
 @p.project_dir
 @p.profiles_dir
 @p.vars
 @p.profile
-@p.state
 @p.target
-@p.target_path
+@p.state
 @p.threads
-@p.full_refresh
+@p.fail_fast
 @requires.postflight
 @requires.preflight
 @requires.profile
 @requires.project
 @requires.runtime_config
+@requires.manifest
 def retry(ctx, **kwargs):
     """Retry the nodes that failed in the previous run."""
-    # Retry will parse manifest inside the task after we consolidate the flags
     task = RetryTask(
         ctx.obj["flags"],
         ctx.obj["runtime_config"],
+        ctx.obj["manifest"],
     )
 
     results = task.run()
@@ -662,7 +613,6 @@ def retry(ctx, **kwargs):
 # dbt clone
 @cli.command("clone")
 @click.pass_context
-@global_flags
 @p.defer_state
 @p.exclude
 @p.full_refresh
@@ -677,6 +627,7 @@ def retry(ctx, **kwargs):
 @p.target_path
 @p.threads
 @p.vars
+@p.version_check
 @requires.preflight
 @requires.profile
 @requires.project
@@ -699,7 +650,6 @@ def clone(ctx, **kwargs):
 # dbt run operation
 @cli.command("run-operation")
 @click.pass_context
-@global_flags
 @click.argument("macro")
 @p.args
 @p.profile
@@ -731,7 +681,6 @@ def run_operation(ctx, **kwargs):
 # dbt seed
 @cli.command("seed")
 @click.pass_context
-@global_flags
 @p.exclude
 @p.full_refresh
 @p.profile
@@ -747,6 +696,7 @@ def run_operation(ctx, **kwargs):
 @p.target_path
 @p.threads
 @p.vars
+@p.version_check
 @requires.postflight
 @requires.preflight
 @requires.profile
@@ -768,7 +718,6 @@ def seed(ctx, **kwargs):
 # dbt snapshot
 @cli.command("snapshot")
 @click.pass_context
-@global_flags
 @p.defer
 @p.deprecated_defer
 @p.exclude
@@ -808,7 +757,6 @@ def snapshot(ctx, **kwargs):
 # dbt source
 @cli.group()
 @click.pass_context
-@global_flags
 def source(ctx, **kwargs):
     """Manage your project's sources"""
 
@@ -816,7 +764,6 @@ def source(ctx, **kwargs):
 # dbt source freshness
 @source.command("freshness")
 @click.pass_context
-@global_flags
 @p.exclude
 @p.output_path  # TODO: Is this ok to re-use?  We have three different output params, how much can we consolidate?
 @p.profile
@@ -859,10 +806,10 @@ cli.commands["source"].add_command(snapshot_freshness, "snapshot-freshness")  #
 # dbt test
 @cli.command("test")
 @click.pass_context
-@global_flags
 @p.defer
 @p.deprecated_defer
 @p.exclude
+@p.fail_fast
 @p.favor_state
 @p.deprecated_favor_state
 @p.indirect_selection
@@ -879,6 +826,7 @@ cli.commands["source"].add_command(snapshot_freshness, "snapshot-freshness")  #
 @p.target_path
 @p.threads
 @p.vars
+@p.version_check
 @requires.postflight
 @requires.preflight
 @requires.profile

core/dbt/cli/option_types.py

@@ -22,26 +22,6 @@ class YAML(ParamType):
             self.fail(f"String '{value}' is not valid YAML", param, ctx)
 
 
-class Package(ParamType):
-    """The Click STRING type. Converts string into dict with package name and version.
-    Example package:
-        package-name@1.0.0
-        package-name
-    """
-
-    name = "NewPackage"
-
-    def convert(self, value, param, ctx):
-        # assume non-string values are a problem
-        if not isinstance(value, str):
-            self.fail(f"Cannot load Package from type {type(value)}", param, ctx)
-        try:
-            package_name, package_version = value.split("@")
-            return {"name": package_name, "version": package_version}
-        except ValueError:
-            return {"name": value, "version": None}
-
-
 class WarnErrorOptionsType(YAML):
     """The Click WarnErrorOptions type. Converts YAML strings into objects."""
 

core/dbt/cli/options.py

@@ -2,21 +2,19 @@ import click
 import inspect
 import typing as t
 from click import Context
-from click.parser import OptionParser, ParsingState
 from dbt.cli.option_types import ChoiceTuple
 
 
 # Implementation from: https://stackoverflow.com/a/48394004
 # Note MultiOption options must be specified with type=tuple or type=ChoiceTuple (https://github.com/pallets/click/issues/2012)
 class MultiOption(click.Option):
-    def __init__(self, *args, **kwargs) -> None:
+    def __init__(self, *args, **kwargs):
         self.save_other_options = kwargs.pop("save_other_options", True)
         nargs = kwargs.pop("nargs", -1)
         assert nargs == -1, "nargs, if set, must be -1 not {}".format(nargs)
         super(MultiOption, self).__init__(*args, **kwargs)
-        # this makes mypy happy, setting these to None causes mypy failures
+        self._previous_parser_process = None
-        self._previous_parser_process = lambda *args, **kwargs: None
+        self._eat_all_parser = None
-        self._eat_all_parser = lambda *args, **kwargs: None
 
         # validate that multiple=True
         multiple = kwargs.pop("multiple", None)
@@ -31,35 +29,34 @@ class MultiOption(click.Option):
         else:
             assert isinstance(option_type, ChoiceTuple), msg
 
-    def add_to_parser(self, parser: OptionParser, ctx: Context):
+    def add_to_parser(self, parser, ctx):
-        def parser_process(value: str, state: ParsingState):
+        def parser_process(value, state):
             # method to hook to the parser.process
             done = False
-            value_list = str.split(value, " ")
+            value = [value]
             if self.save_other_options:
                 # grab everything up to the next option
                 while state.rargs and not done:
-                    for prefix in self._eat_all_parser.prefixes:  # type: ignore[attr-defined]
+                    for prefix in self._eat_all_parser.prefixes:
                         if state.rargs[0].startswith(prefix):
                             done = True
                     if not done:
-                        value_list.append(state.rargs.pop(0))
+                        value.append(state.rargs.pop(0))
             else:
                 # grab everything remaining
-                value_list += state.rargs
+                value += state.rargs
                 state.rargs[:] = []
-            value_tuple = tuple(value_list)
+            value = tuple(value)
             # call the actual process
-            self._previous_parser_process(value_tuple, state)
+            self._previous_parser_process(value, state)
 
         retval = super(MultiOption, self).add_to_parser(parser, ctx)
         for name in self.opts:
             our_parser = parser._long_opt.get(name) or parser._short_opt.get(name)
             if our_parser:
-                self._eat_all_parser = our_parser  # type: ignore[assignment]
+                self._eat_all_parser = our_parser
                 self._previous_parser_process = our_parser.process
-                # mypy doesnt like assingment to a method see https://github.com/python/mypy/issues/708
+                our_parser.process = parser_process
-                our_parser.process = parser_process  # type: ignore[method-assign]
                 break
         return retval
 

core/dbt/cli/params.py

@@ -2,16 +2,10 @@ from pathlib import Path
 
 import click
 from dbt.cli.options import MultiOption
-from dbt.cli.option_types import YAML, ChoiceTuple, WarnErrorOptionsType, Package
+from dbt.cli.option_types import YAML, ChoiceTuple, WarnErrorOptionsType
 from dbt.cli.resolvers import default_project_dir, default_profiles_dir
 from dbt.version import get_version_information
 
-add_package = click.option(
-    "--add-package",
-    help="Add a package to current package spec, specify it as package-name@version. Change the source with --source flag.",
-    envvar=None,
-    type=Package(),
-)
 args = click.option(
     "--args",
     envvar=None,
@@ -46,14 +40,6 @@ compile_docs = click.option(
     default=True,
 )
 
-compile_inject_ephemeral_ctes = click.option(
-    "--inject-ephemeral-ctes/--no-inject-ephemeral-ctes",
-    envvar=None,
-    help="Internal flag controlling injection of referenced ephemeral models' CTEs during `compile`.",
-    hidden=True,
-    default=True,
-)
-
 config_dir = click.option(
     "--config-dir",
     envvar=None,
@@ -83,14 +69,6 @@ deprecated_defer = click.option(
     hidden=True,
 )
 
-dry_run = click.option(
-    "--dry-run",
-    envvar=None,
-    help="Option to run `dbt deps --add-package` without updating package-lock.yml file.",
-    is_flag=True,
-)
-
-
 enable_legacy_logger = click.option(
     "--enable-legacy-logger/--no-enable-legacy-logger",
     envvar="DBT_ENABLE_LEGACY_LOGGER",
@@ -133,14 +111,6 @@ full_refresh = click.option(
     is_flag=True,
 )
 
-host = click.option(
-    "--host",
-    envvar="DBT_HOST",
-    help="host to serve dbt docs on",
-    type=click.STRING,
-    default="127.0.0.1",
-)
-
 indirect_selection = click.option(
     "--indirect-selection",
     envvar="DBT_INDIRECT_SELECTION",
@@ -149,13 +119,6 @@ indirect_selection = click.option(
     default="eager",
 )
 
-lock = click.option(
-    "--lock",
-    envvar=None,
-    help="Generate the package-lock.yml file without install the packages.",
-    is_flag=True,
-)
-
 log_cache_events = click.option(
     "--log-cache-events/--no-log-cache-events",
     help="Enable verbose logging for relational cache events to help when debugging.",
@@ -294,14 +257,6 @@ partial_parse_file_path = click.option(
     type=click.Path(exists=True, dir_okay=False, resolve_path=True),
 )
 
-partial_parse_file_diff = click.option(
-    "--partial-parse-file-diff/--no-partial-parse-file-diff",
-    envvar="DBT_PARTIAL_PARSE_FILE_DIFF",
-    help="Internal flag for whether to compute a file diff during partial parsing.",
-    hidden=True,
-    default=True,
-)
-
 populate_cache = click.option(
     "--populate-cache/--no-populate-cache",
     envvar="DBT_POPULATE_CACHE",
@@ -344,7 +299,7 @@ printer_width = click.option(
 profile = click.option(
     "--profile",
     envvar=None,
-    help="Which existing profile to load. Overrides setting in dbt_project.yml.",
+    help="Which profile to load. Overrides setting in dbt_project.yml.",
 )
 
 profiles_dir = click.option(
@@ -397,8 +352,6 @@ resource_type = click.option(
     type=ChoiceTuple(
         [
             "metric",
-            "semantic_model",
-            "saved_query",
             "source",
             "analysis",
             "model",
@@ -416,14 +369,6 @@ resource_type = click.option(
     default=(),
 )
 
-include_saved_query = click.option(
-    "--include-saved-query/--no-include-saved-query",
-    envvar="DBT_INCLUDE_SAVED_QUERY",
-    help="Include saved queries in the list of resources to be selected for build command",
-    is_flag=True,
-    hidden=True,
-)
-
 model_decls = ("-m", "--models", "--model")
 select_decls = ("-s", "--select")
 select_attrs = {
@@ -444,9 +389,9 @@ inline = click.option(
 # Most CLI arguments should use the combined `select` option that aliases `--models` to `--select`.
 # However, if you need to split out these separators (like `dbt ls`), use the `models` and `raw_select` options instead.
 # See https://github.com/dbt-labs/dbt-core/pull/6774#issuecomment-1408476095 for more info.
-models = click.option(*model_decls, **select_attrs)  # type: ignore[arg-type]
+models = click.option(*model_decls, **select_attrs)
-raw_select = click.option(*select_decls, **select_attrs)  # type: ignore[arg-type]
+raw_select = click.option(*select_decls, **select_attrs)
-select = click.option(*select_decls, *model_decls, **select_attrs)  # type: ignore[arg-type]
+select = click.option(*select_decls, *model_decls, **select_attrs)
 
 selector = click.option(
     "--selector",
@@ -461,13 +406,6 @@ send_anonymous_usage_stats = click.option(
     default=True,
 )
 
-clean_project_files_only = click.option(
-    "--clean-project-files-only / --no-clean-project-files-only",
-    envvar="DBT_CLEAN_PROJECT_FILES_ONLY",
-    help="If disabled, dbt clean will delete all paths specified in clean-paths, even if they're outside the dbt project.",
-    default=True,
-)
-
 show = click.option(
     "--show",
     envvar=None,
@@ -503,21 +441,6 @@ empty_catalog = click.option(
     is_flag=True,
 )
 
-source = click.option(
-    "--source",
-    envvar=None,
-    help="Source to download page from, must be one of hub, git, or local. Defaults to hub.",
-    type=click.Choice(["hub", "git", "local"], case_sensitive=True),
-    default="hub",
-)
-
-static = click.option(
-    "--static",
-    help="Generate an additional static_index.html with manifest and catalog built-in.",
-    default=False,
-    is_flag=True,
-)
-
 state = click.option(
     "--state",
     envvar="DBT_STATE",
@@ -586,13 +509,6 @@ target_path = click.option(
     type=click.Path(),
 )
 
-upgrade = click.option(
-    "--upgrade",
-    envvar=None,
-    help="Upgrade packages to the latest version.",
-    is_flag=True,
-)
-
 debug_connection = click.option(
     "--connection",
     envvar=None,
@@ -674,10 +590,3 @@ write_json = click.option(
     help="Whether or not to write the manifest.json and run_results.json files to the target directory",
     default=True,
 )
-
-show_resource_report = click.option(
-    "--show-resource-report/--no-show-resource-report",
-    default=False,
-    envvar="DBT_SHOW_RESOURCE_REPORT",
-    hidden=True,
-)

core/dbt/cli/requires.py

@@ -1,6 +1,6 @@
 import dbt.tracking
 from dbt.version import installed as installed_version
-from dbt.adapters.factory import adapter_management, register_adapter, get_adapter
+from dbt.adapters.factory import adapter_management, register_adapter
 from dbt.flags import set_flags, get_flag_dict
 from dbt.cli.exceptions import (
     ExceptionExit,
@@ -9,28 +9,24 @@ from dbt.cli.exceptions import (
 from dbt.cli.flags import Flags
 from dbt.config import RuntimeConfig
 from dbt.config.runtime import load_project, load_profile, UnsetProfile
-from dbt.events.base_types import EventLevel
 from dbt.events.functions import fire_event, LOG_VERSION, set_invocation_id, setup_event_logger
 from dbt.events.types import (
     CommandCompleted,
     MainReportVersion,
     MainReportArgs,
     MainTrackingUserState,
-    ResourceReport,
 )
 from dbt.events.helpers import get_json_string_utcnow
 from dbt.events.types import MainEncounteredError, MainStackTrace
 from dbt.exceptions import Exception as DbtException, DbtProjectError, FailFastError
-from dbt.parser.manifest import parse_manifest
+from dbt.parser.manifest import ManifestLoader, write_manifest
 from dbt.profiler import profiler
 from dbt.tracking import active_user, initialize_from_flags, track_run
 from dbt.utils import cast_dict_to_dict_of_strings
-from dbt.plugins import set_up_plugin_manager
+from dbt.plugins import set_up_plugin_manager, get_plugin_manager
-
 
 from click import Context
 from functools import update_wrapper
-import importlib.util
 import time
 import traceback
 
@@ -100,28 +96,6 @@ def postflight(func):
             fire_event(MainStackTrace(stack_trace=traceback.format_exc()))
             raise ExceptionExit(e)
         finally:
-            # Fire ResourceReport, but only on systems which support the resource
-            # module. (Skip it on Windows).
-            if importlib.util.find_spec("resource") is not None:
-                import resource
-
-                rusage = resource.getrusage(resource.RUSAGE_SELF)
-                fire_event(
-                    ResourceReport(
-                        command_name=ctx.command.name,
-                        command_success=success,
-                        command_wall_clock_time=time.perf_counter() - start_func,
-                        process_user_time=rusage.ru_utime,
-                        process_kernel_time=rusage.ru_stime,
-                        process_mem_max_rss=rusage.ru_maxrss,
-                        process_in_blocks=rusage.ru_inblock,
-                        process_out_blocks=rusage.ru_oublock,
-                    ),
-                    EventLevel.INFO
-                    if "flags" in ctx.obj and ctx.obj["flags"].SHOW_RESOURCE_REPORT
-                    else None,
-                )
-
             fire_event(
                 CommandCompleted(
                     command=ctx.command_path,
@@ -265,16 +239,23 @@ def manifest(*args0, write=True, write_perf_info=False):
                 raise DbtProjectError("profile, project, and runtime_config required for manifest")
 
             runtime_config = ctx.obj["runtime_config"]
+            register_adapter(runtime_config)
 
             # a manifest has already been set on the context, so don't overwrite it
             if ctx.obj.get("manifest") is None:
-                ctx.obj["manifest"] = parse_manifest(
+                manifest = ManifestLoader.get_full_manifest(
-                    runtime_config, write_perf_info, write, ctx.obj["flags"].write_json
+                    runtime_config,
+                    write_perf_info=write_perf_info,
                 )
-            else:
+
-                register_adapter(runtime_config)
+                ctx.obj["manifest"] = manifest
-                adapter = get_adapter(runtime_config)
+                if write and ctx.obj["flags"].write_json:
-                adapter.connections.set_query_header(ctx.obj["manifest"])
+                    write_manifest(manifest, runtime_config.project_target_path)
+                    pm = get_plugin_manager(runtime_config.project_name)
+                    plugin_artifacts = pm.get_manifest_artifacts(manifest)
+                    for path, plugin_artifact in plugin_artifacts.items():
+                        plugin_artifact.write(path)
+
             return func(*args, **kwargs)
 
         return update_wrapper(wrapper, func)

core/dbt/clients/agate_helper.py

@@ -9,23 +9,10 @@ from typing import Iterable, List, Dict, Union, Optional, Any
 
 from dbt.exceptions import DbtRuntimeError
 
+
 BOM = BOM_UTF8.decode("utf-8")  # '\ufeff'
 
 
-class Integer(agate.data_types.DataType):
-    def cast(self, d):
-        # by default agate will cast none as a Number
-        # but we need to cast it as an Integer to preserve
-        # the type when merging and unioning tables
-        if type(d) == int or d is None:
-            return d
-        else:
-            raise agate.exceptions.CastError('Can not parse value "%s" as Integer.' % d)
-
-    def jsonify(self, d):
-        return d
-
-
 class Number(agate.data_types.Number):
     # undo the change in https://github.com/wireservice/agate/pull/733
     # i.e. do not cast True and False to numeric 1 and 0
@@ -61,7 +48,6 @@ def build_type_tester(
 ) -> agate.TypeTester:
 
     types = [
-        Integer(null_values=("null", "")),
         Number(null_values=("null", "")),
         agate.data_types.Date(null_values=("null", ""), date_format="%Y-%m-%d"),
         agate.data_types.DateTime(null_values=("null", ""), datetime_format="%Y-%m-%d %H:%M:%S"),
@@ -149,12 +135,12 @@ def as_matrix(table):
     return [r.values() for r in table.rows.values()]
 
 
-def from_csv(abspath, text_columns, delimiter=","):
+def from_csv(abspath, text_columns):
     type_tester = build_type_tester(text_columns=text_columns)
     with open(abspath, encoding="utf-8") as fp:
         if fp.read(1) != BOM:
             fp.seek(0)
-        return agate.Table.from_csv(fp, column_types=type_tester, delimiter=delimiter)
+        return agate.Table.from_csv(fp, column_types=type_tester)
 
 
 class _NullMarker:
@@ -165,7 +151,7 @@ NullableAgateType = Union[agate.data_types.DataType, _NullMarker]
 
 
 class ColumnTypeBuilder(Dict[str, NullableAgateType]):
-    def __init__(self) -> None:
+    def __init__(self):
         super().__init__()
 
     def __setitem__(self, key, value):
@@ -180,13 +166,6 @@ class ColumnTypeBuilder(Dict[str, NullableAgateType]):
         elif isinstance(value, _NullMarker):
             # use the existing value
             return
-        # when one table column is Number while another is Integer, force the column to Number on merge
-        elif isinstance(value, Integer) and isinstance(existing_type, agate.data_types.Number):
-            # use the existing value
-            return
-        elif isinstance(existing_type, Integer) and isinstance(value, agate.data_types.Number):
-            # overwrite
-            super().__setitem__(key, value)
         elif not isinstance(value, type(existing_type)):
             # actual type mismatch!
             raise DbtRuntimeError(
@@ -198,9 +177,8 @@ class ColumnTypeBuilder(Dict[str, NullableAgateType]):
         result: Dict[str, agate.data_types.DataType] = {}
         for key, value in self.items():
             if isinstance(value, _NullMarker):
-                # agate would make it a Number but we'll make it Integer so that if this column
+                # this is what agate would do.
-                # gets merged with another Integer column, it won't get forced to a Number
+                result[key] = agate.data_types.Number()
-                result[key] = Integer()
             else:
                 result[key] = value
         return result
@@ -240,12 +218,3 @@ def merge_tables(tables: List[agate.Table]) -> agate.Table:
                 rows.append(agate.Row(data, column_names))
     # _is_fork to tell agate that we already made things into `Row`s.
     return agate.Table(rows, column_names, column_types, _is_fork=True)
-
-
-def get_column_value_uncased(column_name: str, row: agate.Row) -> Any:
-    """Get the value of a column in this row, ignoring the casing of the column name."""
-    for key, value in row.items():
-        if key.casefold() == column_name.casefold():
-            return value
-
-    raise KeyError

core/dbt/clients/git.py

@@ -111,7 +111,7 @@ def checkout(cwd, repo, revision=None):
 def get_current_sha(cwd):
     out, err = run_cmd(cwd, ["git", "rev-parse", "HEAD"], env={"LC_ALL": "C"})
 
-    return out.decode("utf-8").strip()
+    return out.decode("utf-8")
 
 
 def remove_remote(cwd):

core/dbt/clients/jinja.py

@@ -191,7 +191,7 @@ NativeSandboxEnvironment.template_class = NativeSandboxTemplate  # type: ignore
 
 
 class TemplateCache:
-    def __init__(self) -> None:
+    def __init__(self):
         self.file_cache: Dict[str, jinja2.Template] = {}
 
     def get_node_template(self, node) -> jinja2.Template:

core/dbt/compilation.py

@@ -125,7 +125,7 @@ def _get_tests_for_node(manifest: Manifest, unique_id: UniqueID) -> List[UniqueI
 
 
 class Linker:
-    def __init__(self, data=None) -> None:
+    def __init__(self, data=None):
         if data is None:
             data = {}
         self.graph = nx.DiGraph(**data)
@@ -183,16 +183,14 @@ class Linker:
     def link_graph(self, manifest: Manifest):
         for source in manifest.sources.values():
             self.add_node(source.unique_id)
+        for semantic_model in manifest.semantic_models.values():
+            self.add_node(semantic_model.unique_id)
         for node in manifest.nodes.values():
             self.link_node(node, manifest)
-        for semantic_model in manifest.semantic_models.values():
-            self.link_node(semantic_model, manifest)
         for exposure in manifest.exposures.values():
             self.link_node(exposure, manifest)
         for metric in manifest.metrics.values():
             self.link_node(metric, manifest)
-        for saved_query in manifest.saved_queries.values():
-            self.link_node(saved_query, manifest)
 
         cycle = self.find_cycles()
 
@@ -276,7 +274,7 @@ class Linker:
 
 
 class Compiler:
-    def __init__(self, config) -> None:
+    def __init__(self, config):
         self.config = config
 
     def initialize(self):
@@ -322,10 +320,6 @@ class Compiler:
         if model.compiled_code is None:
             raise DbtRuntimeError("Cannot inject ctes into an uncompiled node", model)
 
-        # tech debt: safe flag/arg access (#6259)
-        if not getattr(self.config.args, "inject_ephemeral_ctes", True):
-            return (model, [])
-
         # extra_ctes_injected flag says that we've already recursively injected the ctes
         if model.extra_ctes_injected:
             return (model, model.extra_ctes)

core/dbt/config/__init__.py

@@ -1,4 +1,4 @@
 # all these are just exports, they need "noqa" so flake8 will not complain.
-from .profile import Profile  # noqa
+from .profile import Profile, read_user_config  # noqa
 from .project import Project, IsFQNResource, PartialProject  # noqa
 from .runtime import RuntimeConfig  # noqa

core/dbt/config/profile.py

@@ -8,7 +8,7 @@ from dbt.flags import get_flags
 from dbt.clients.system import load_file_contents
 from dbt.clients.yaml_helper import load_yaml_text
 from dbt.contracts.connection import Credentials, HasCredentials
-from dbt.contracts.project import ProfileConfig
+from dbt.contracts.project import ProfileConfig, UserConfig
 from dbt.exceptions import (
     CompilationError,
     DbtProfileError,
@@ -19,6 +19,7 @@ from dbt.exceptions import (
 )
 from dbt.events.types import MissingProfileTarget
 from dbt.events.functions import fire_event
+from dbt.utils import coerce_dict_str
 
 from .renderer import ProfileRenderer
 
@@ -50,6 +51,19 @@ def read_profile(profiles_dir: str) -> Dict[str, Any]:
     return {}
 
 
+def read_user_config(directory: str) -> UserConfig:
+    try:
+        profile = read_profile(directory)
+        if profile:
+            user_config = coerce_dict_str(profile.get("config", {}))
+            if user_config is not None:
+                UserConfig.validate(user_config)
+                return UserConfig.from_dict(user_config)
+    except (DbtRuntimeError, ValidationError):
+        pass
+    return UserConfig()
+
+
 # The Profile class is included in RuntimeConfig, so any attribute
 # additions must also be set where the RuntimeConfig class is created
 # `init=False` is a workaround for https://bugs.python.org/issue45081
@@ -57,6 +71,7 @@ def read_profile(profiles_dir: str) -> Dict[str, Any]:
 class Profile(HasCredentials):
     profile_name: str
     target_name: str
+    user_config: UserConfig
     threads: int
     credentials: Credentials
     profile_env_vars: Dict[str, Any]
@@ -65,14 +80,16 @@ class Profile(HasCredentials):
         self,
         profile_name: str,
         target_name: str,
+        user_config: UserConfig,
         threads: int,
         credentials: Credentials,
-    ) -> None:
+    ):
         """Explicitly defining `__init__` to work around bug in Python 3.9.7
         https://bugs.python.org/issue45081
         """
         self.profile_name = profile_name
         self.target_name = target_name
+        self.user_config = user_config
         self.threads = threads
         self.credentials = credentials
         self.profile_env_vars = {}  # never available on init
@@ -89,10 +106,12 @@ class Profile(HasCredentials):
         result = {
             "profile_name": self.profile_name,
             "target_name": self.target_name,
+            "user_config": self.user_config,
             "threads": self.threads,
             "credentials": self.credentials,
         }
         if serialize_credentials:
+            result["user_config"] = self.user_config.to_dict(omit_none=True)
             result["credentials"] = self.credentials.to_dict(omit_none=True)
         return result
 
@@ -105,6 +124,7 @@ class Profile(HasCredentials):
                 "name": self.target_name,
                 "target_name": self.target_name,
                 "profile_name": self.profile_name,
+                "config": self.user_config.to_dict(omit_none=True),
             }
         )
         return target
@@ -226,6 +246,7 @@ defined in your profiles.yml file. You can find profiles.yml here:
         threads: int,
         profile_name: str,
         target_name: str,
+        user_config: Optional[Dict[str, Any]] = None,
     ) -> "Profile":
         """Create a profile from an existing set of Credentials and the
         remaining information.
@@ -234,13 +255,20 @@ defined in your profiles.yml file. You can find profiles.yml here:
         :param threads: The number of threads to use for connections.
         :param profile_name: The profile name used for this profile.
         :param target_name: The target name used for this profile.
+        :param user_config: The user-level config block from the
+            raw profiles, if specified.
         :raises DbtProfileError: If the profile is invalid.
         :returns: The new Profile object.
         """
+        if user_config is None:
+            user_config = {}
+        UserConfig.validate(user_config)
+        user_config_obj: UserConfig = UserConfig.from_dict(user_config)
 
         profile = cls(
             profile_name=profile_name,
             target_name=target_name,
+            user_config=user_config_obj,
             threads=threads,
             credentials=credentials,
         )
@@ -288,6 +316,7 @@ defined in your profiles.yml file. You can find profiles.yml here:
         raw_profile: Dict[str, Any],
         profile_name: str,
         renderer: ProfileRenderer,
+        user_config: Optional[Dict[str, Any]] = None,
         target_override: Optional[str] = None,
         threads_override: Optional[int] = None,
     ) -> "Profile":
@@ -299,6 +328,8 @@ defined in your profiles.yml file. You can find profiles.yml here:
             disk as yaml and its values rendered with jinja.
         :param profile_name: The profile name used.
         :param renderer: The config renderer.
+        :param user_config: The global config for the user, if it
+            was present.
         :param target_override: The target to use, if provided on
             the command line.
         :param threads_override: The thread count to use, if
@@ -307,6 +338,9 @@ defined in your profiles.yml file. You can find profiles.yml here:
             target could not be found
         :returns: The new Profile object.
         """
+        # user_config is not rendered.
+        if user_config is None:
+            user_config = raw_profile.get("config")
         # TODO: should it be, and the values coerced to bool?
         target_name, profile_data = cls.render_profile(
             raw_profile, profile_name, target_override, renderer
@@ -327,6 +361,7 @@ defined in your profiles.yml file. You can find profiles.yml here:
             profile_name=profile_name,
             target_name=target_name,
             threads=threads,
+            user_config=user_config,
         )
 
     @classmethod
@@ -361,11 +396,13 @@ defined in your profiles.yml file. You can find profiles.yml here:
         if not raw_profile:
             msg = f"Profile {profile_name} in profiles.yml is empty"
             raise DbtProfileError(INVALID_PROFILE_MESSAGE.format(error_string=msg))
+        user_config = raw_profiles.get("config")
 
         return cls.from_raw_profile_info(
             raw_profile=raw_profile,
             profile_name=profile_name,
             renderer=renderer,
+            user_config=user_config,
             target_override=target_override,
             threads_override=threads_override,
         )

core/dbt/config/project.py

@@ -16,13 +16,8 @@ import os
 
 from dbt.flags import get_flags
 from dbt import deprecations
-from dbt.constants import (
+from dbt.constants import DEPENDENCIES_FILE_NAME, PACKAGES_FILE_NAME
-    DEPENDENCIES_FILE_NAME,
+from dbt.clients.system import path_exists, resolve_path_from_base, load_file_contents
-    PACKAGES_FILE_NAME,
-    PACKAGE_LOCK_HASH_KEY,
-    DBT_PROJECT_FILE_NAME,
-)
-from dbt.clients.system import path_exists, load_file_contents
 from dbt.clients.yaml_helper import load_yaml_text
 from dbt.contracts.connection import QueryComment
 from dbt.exceptions import (
@@ -36,13 +31,12 @@ from dbt.graph import SelectionSpec
 from dbt.helper_types import NoValue
 from dbt.semver import VersionSpecifier, versions_compatible
 from dbt.version import get_installed_version
-from dbt.utils import MultiDict, md5, coerce_dict_str
+from dbt.utils import MultiDict, md5
 from dbt.node_types import NodeType
 from dbt.config.selectors import SelectorDict
 from dbt.contracts.project import (
     Project as ProjectContract,
     SemverString,
-    ProjectFlags,
 )
 from dbt.contracts.project import PackageConfig, ProjectPackageMetadata
 from dbt.dataclass_schema import ValidationError
@@ -83,8 +77,8 @@ Validator Error:
 """
 
 MISSING_DBT_PROJECT_ERROR = """\
-No {DBT_PROJECT_FILE_NAME} found at expected path {path}
+No dbt_project.yml found at expected path {path}
-Verify that each entry within packages.yml (and their transitive dependencies) contains a file named {DBT_PROJECT_FILE_NAME}
+Verify that each entry within packages.yml (and their transitive dependencies) contains a file named dbt_project.yml
 """
 
 
@@ -100,17 +94,16 @@ def _load_yaml(path):
     return load_yaml_text(contents)
 
 
-def load_yml_dict(file_path):
-    ret = {}
-    if path_exists(file_path):
-        ret = _load_yaml(file_path) or {}
-    return ret
-
-
 def package_and_project_data_from_root(project_root):
+    package_filepath = resolve_path_from_base(PACKAGES_FILE_NAME, project_root)
+    dependencies_filepath = resolve_path_from_base(DEPENDENCIES_FILE_NAME, project_root)
 
-    packages_yml_dict = load_yml_dict(f"{project_root}/{PACKAGES_FILE_NAME}")
+    packages_yml_dict = {}
-    dependencies_yml_dict = load_yml_dict(f"{project_root}/{DEPENDENCIES_FILE_NAME}")
+    dependencies_yml_dict = {}
+    if path_exists(package_filepath):
+        packages_yml_dict = _load_yaml(package_filepath) or {}
+    if path_exists(dependencies_filepath):
+        dependencies_yml_dict = _load_yaml(dependencies_filepath) or {}
 
     if "packages" in packages_yml_dict and "packages" in dependencies_yml_dict:
         msg = "The 'packages' key cannot be specified in both packages.yml and dependencies.yml"
@@ -130,21 +123,10 @@ def package_and_project_data_from_root(project_root):
     return packages_dict, packages_specified_path
 
 
-def package_config_from_data(
+def package_config_from_data(packages_data: Dict[str, Any]) -> PackageConfig:
-    packages_data: Dict[str, Any],
-    unrendered_packages_data: Optional[Dict[str, Any]] = None,
-) -> PackageConfig:
     if not packages_data:
         packages_data = {"packages": []}
 
-    # this depends on the two lists being in the same order
-    if unrendered_packages_data:
-        unrendered_packages_data = deepcopy(unrendered_packages_data)
-        for i in range(0, len(packages_data.get("packages", []))):
-            packages_data["packages"][i]["unrendered"] = unrendered_packages_data["packages"][i]
-
-    if PACKAGE_LOCK_HASH_KEY in packages_data:
-        packages_data.pop(PACKAGE_LOCK_HASH_KEY)
     try:
         PackageConfig.validate(packages_data)
         packages = PackageConfig.from_dict(packages_data)
@@ -201,20 +183,16 @@ def value_or(value: Optional[T], default: T) -> T:
 def load_raw_project(project_root: str) -> Dict[str, Any]:
 
     project_root = os.path.normpath(project_root)
-    project_yaml_filepath = os.path.join(project_root, DBT_PROJECT_FILE_NAME)
+    project_yaml_filepath = os.path.join(project_root, "dbt_project.yml")
 
     # get the project.yml contents
     if not path_exists(project_yaml_filepath):
-        raise DbtProjectError(
+        raise DbtProjectError(MISSING_DBT_PROJECT_ERROR.format(path=project_yaml_filepath))
-            MISSING_DBT_PROJECT_ERROR.format(
-                path=project_yaml_filepath, DBT_PROJECT_FILE_NAME=DBT_PROJECT_FILE_NAME
-            )
-        )
 
     project_dict = _load_yaml(project_yaml_filepath)
 
     if not isinstance(project_dict, dict):
-        raise DbtProjectError(f"{DBT_PROJECT_FILE_NAME} does not parse to a dictionary")
+        raise DbtProjectError("dbt_project.yml does not parse to a dictionary")
 
     return project_dict
 
@@ -329,21 +307,21 @@ class PartialProject(RenderComponents):
             selectors_dict=rendered_selectors,
         )
 
-    # Called by Project.from_project_root which first calls PartialProject.from_project_root
+    # Called by Project.from_project_root (not PartialProject.from_project_root!)
     def render(self, renderer: DbtProjectYamlRenderer) -> "Project":
         try:
             rendered = self.get_rendered(renderer)
             return self.create_project(rendered)
         except DbtProjectError as exc:
             if exc.path is None:
-                exc.path = os.path.join(self.project_root, DBT_PROJECT_FILE_NAME)
+                exc.path = os.path.join(self.project_root, "dbt_project.yml")
             raise
 
     def render_package_metadata(self, renderer: PackageRenderer) -> ProjectPackageMetadata:
         packages_data = renderer.render_data(self.packages_dict)
-        packages_config = package_config_from_data(packages_data, self.packages_dict)
+        packages_config = package_config_from_data(packages_data)
         if not self.project_name:
-            raise DbtProjectError(f"Package defined in {DBT_PROJECT_FILE_NAME} must have a name!")
+            raise DbtProjectError("Package dbt_project.yml must have a name!")
         return ProjectPackageMetadata(self.project_name, packages_config.packages)
 
     def check_config_path(
@@ -354,7 +332,7 @@ class PartialProject(RenderComponents):
                 msg = (
                     "{deprecated_path} and {expected_path} cannot both be defined. The "
                     "`{deprecated_path}` config has been deprecated in favor of `{expected_path}`. "
-                    f"Please update your `{DBT_PROJECT_FILE_NAME}` configuration to reflect this "
+                    "Please update your `dbt_project.yml` configuration to reflect this "
                     "change."
                 )
                 raise DbtProjectError(
@@ -426,11 +404,11 @@ class PartialProject(RenderComponents):
 
         docs_paths: List[str] = value_or(cfg.docs_paths, all_source_paths)
         asset_paths: List[str] = value_or(cfg.asset_paths, [])
-        global_flags = get_flags()
+        flags = get_flags()
 
-        flag_target_path = str(global_flags.TARGET_PATH) if global_flags.TARGET_PATH else None
+        flag_target_path = str(flags.TARGET_PATH) if flags.TARGET_PATH else None
         target_path: str = flag_or(flag_target_path, cfg.target_path, "target")
-        log_path: str = str(global_flags.LOG_PATH)
+        log_path: str = str(flags.LOG_PATH)
 
         clean_targets: List[str] = value_or(cfg.clean_targets, [target_path])
         packages_install_path: str = value_or(cfg.packages_install_path, "dbt_packages")
@@ -448,11 +426,8 @@ class PartialProject(RenderComponents):
         sources: Dict[str, Any]
         tests: Dict[str, Any]
         metrics: Dict[str, Any]
-        semantic_models: Dict[str, Any]
-        saved_queries: Dict[str, Any]
         exposures: Dict[str, Any]
         vars_value: VarProvider
-        dbt_cloud: Dict[str, Any]
 
         dispatch = cfg.dispatch
         models = cfg.models
@@ -461,8 +436,6 @@ class PartialProject(RenderComponents):
         sources = cfg.sources
         tests = cfg.tests
         metrics = cfg.metrics
-        semantic_models = cfg.semantic_models
-        saved_queries = cfg.saved_queries
         exposures = cfg.exposures
         if cfg.vars is None:
             vars_dict: Dict[str, Any] = {}
@@ -476,9 +449,8 @@ class PartialProject(RenderComponents):
         on_run_end: List[str] = value_or(cfg.on_run_end, [])
 
         query_comment = _query_comment_from_cfg(cfg.query_comment)
-        packages: PackageConfig = package_config_from_data(
+
-            rendered.packages_dict, unrendered.packages_dict
+        packages: PackageConfig = package_config_from_data(rendered.packages_dict)
-        )
         selectors = selector_config_from_data(rendered.selectors_dict)
         manifest_selectors: Dict[str, Any] = {}
         if rendered.selectors_dict and rendered.selectors_dict["selectors"]:
@@ -487,8 +459,6 @@ class PartialProject(RenderComponents):
             manifest_selectors = SelectorDict.parse_from_selectors_list(
                 rendered.selectors_dict["selectors"]
             )
-        dbt_cloud = cfg.dbt_cloud
-
         project = Project(
             project_name=name,
             version=version,
@@ -522,15 +492,12 @@ class PartialProject(RenderComponents):
             sources=sources,
             tests=tests,
             metrics=metrics,
-            semantic_models=semantic_models,
-            saved_queries=saved_queries,
             exposures=exposures,
             vars=vars_value,
             config_version=cfg.config_version,
             unrendered=unrendered,
             project_env_vars=project_env_vars,
             restrict_access=cfg.restrict_access,
-            dbt_cloud=dbt_cloud,
         )
         # sanity check - this means an internal issue
         project.validate()
@@ -574,12 +541,6 @@ class PartialProject(RenderComponents):
             packages_specified_path,
         ) = package_and_project_data_from_root(project_root)
         selectors_dict = selector_data_from_root(project_root)
-
-        if "flags" in project_dict:
-            # We don't want to include "flags" in the Project,
-            # it goes in ProjectFlags
-            project_dict.pop("flags")
-
         return cls.from_dicts(
             project_root=project_root,
             project_dict=project_dict,
@@ -637,8 +598,6 @@ class Project:
     sources: Dict[str, Any]
     tests: Dict[str, Any]
     metrics: Dict[str, Any]
-    semantic_models: Dict[str, Any]
-    saved_queries: Dict[str, Any]
     exposures: Dict[str, Any]
     vars: VarProvider
     dbt_version: List[VersionSpecifier]
@@ -650,7 +609,6 @@ class Project:
     unrendered: RenderComponents
     project_env_vars: Dict[str, Any]
     restrict_access: bool
-    dbt_cloud: Dict[str, Any]
 
     @property
     def all_source_paths(self) -> List[str]:
@@ -715,13 +673,11 @@ class Project:
                 "sources": self.sources,
                 "tests": self.tests,
                 "metrics": self.metrics,
-                "semantic-models": self.semantic_models,
-                "saved-queries": self.saved_queries,
                 "exposures": self.exposures,
                 "vars": self.vars.to_dict(),
                 "require-dbt-version": [v.to_version_string() for v in self.dbt_version],
+                "config-version": self.config_version,
                 "restrict-access": self.restrict_access,
-                "dbt-cloud": self.dbt_cloud,
             }
         )
         if self.query_comment:
@@ -783,52 +739,3 @@ class Project:
     def project_target_path(self):
         # If target_path is absolute, project_root will not be included
         return os.path.join(self.project_root, self.target_path)
-
-
-def read_project_flags(project_dir: str, profiles_dir: str) -> ProjectFlags:
-    try:
-        project_flags: Dict[str, Any] = {}
-        # Read project_flags from dbt_project.yml first
-        # Flags are instantiated before the project, so we don't
-        # want to throw an error for non-existence of dbt_project.yml here
-        # because it breaks things.
-        project_root = os.path.normpath(project_dir)
-        project_yaml_filepath = os.path.join(project_root, DBT_PROJECT_FILE_NAME)
-        if path_exists(project_yaml_filepath):
-            try:
-                project_dict = load_raw_project(project_root)
-                if "flags" in project_dict:
-                    project_flags = project_dict.pop("flags")
-            except Exception:
-                # This is probably a yaml load error.The error will be reported
-                # later, when the project loads.
-                pass
-
-        from dbt.config.profile import read_profile
-
-        profile = read_profile(profiles_dir)
-        profile_project_flags: Optional[Dict[str, Any]] = {}
-        if profile:
-            profile_project_flags = coerce_dict_str(profile.get("config", {}))
-
-        if project_flags and profile_project_flags:
-            raise DbtProjectError(
-                f"Do not specify both 'config' in profiles.yml and 'flags' in {DBT_PROJECT_FILE_NAME}. "
-                "Using 'config' in profiles.yml is deprecated."
-            )
-
-        if profile_project_flags:
-            # This can't use WARN_ERROR or WARN_ERROR_OPTIONS because they're in
-            # the config that we're loading. Uses special "buffer" method and fired after flags are initialized in preflight.
-            deprecations.buffer("project-flags-moved")
-            project_flags = profile_project_flags
-
-        if project_flags is not None:
-            ProjectFlags.validate(project_flags)
-            return ProjectFlags.from_dict(project_flags)
-    except (DbtProjectError) as exc:
-        # We don't want to eat the DbtProjectError for UserConfig to ProjectFlags
-        raise exc
-    except (DbtRuntimeError, ValidationError):
-        pass
-    return ProjectFlags()

core/dbt/config/renderer.py

@@ -74,7 +74,7 @@ def _list_if_none_or_string(value):
 
 
 class ProjectPostprocessor(Dict[Keypath, Callable[[Any], Any]]):
-    def __init__(self) -> None:
+    def __init__(self):
         super().__init__()
 
         self[("on-run-start",)] = _list_if_none_or_string

core/dbt/config/runtime.py

@@ -20,7 +20,7 @@ from dbt.adapters.factory import get_include_paths, get_relation_class_by_name
 from dbt.config.project import load_raw_project
 from dbt.contracts.connection import AdapterRequiredConfig, Credentials, HasCredentials
 from dbt.contracts.graph.manifest import ManifestMetadata
-from dbt.contracts.project import Configuration
+from dbt.contracts.project import Configuration, UserConfig
 from dbt.contracts.relation import ComponentName
 from dbt.dataclass_schema import ValidationError
 from dbt.events.functions import warn_or_error
@@ -167,8 +167,6 @@ class RuntimeConfig(Project, Profile, AdapterRequiredConfig):
             sources=project.sources,
             tests=project.tests,
             metrics=project.metrics,
-            semantic_models=project.semantic_models,
-            saved_queries=project.saved_queries,
             exposures=project.exposures,
             vars=project.vars,
             config_version=project.config_version,
@@ -178,12 +176,12 @@ class RuntimeConfig(Project, Profile, AdapterRequiredConfig):
             profile_env_vars=profile.profile_env_vars,
             profile_name=profile.profile_name,
             target_name=profile.target_name,
+            user_config=profile.user_config,
             threads=profile.threads,
             credentials=profile.credentials,
             args=args,
             cli_vars=cli_vars,
             dependencies=dependencies,
-            dbt_cloud=project.dbt_cloud,
         )
 
     # Called by 'load_projects' in this class
@@ -324,8 +322,6 @@ class RuntimeConfig(Project, Profile, AdapterRequiredConfig):
             "sources": self._get_config_paths(self.sources),
             "tests": self._get_config_paths(self.tests),
             "metrics": self._get_config_paths(self.metrics),
-            "semantic_models": self._get_config_paths(self.semantic_models),
-            "saved_queries": self._get_config_paths(self.saved_queries),
             "exposures": self._get_config_paths(self.exposures),
         }
 
@@ -408,7 +404,7 @@ class RuntimeConfig(Project, Profile, AdapterRequiredConfig):
 
 
 class UnsetCredentials(Credentials):
-    def __init__(self) -> None:
+    def __init__(self):
         super().__init__("", "")
 
     @property
@@ -431,6 +427,7 @@ class UnsetCredentials(Credentials):
 class UnsetProfile(Profile):
     def __init__(self):
         self.credentials = UnsetCredentials()
+        self.user_config = UserConfig()  # This will be read in _get_rendered_profile
         self.profile_name = ""
         self.target_name = ""
         self.threads = -1

core/dbt/constants.py

@@ -1,4 +1,4 @@
-SECRET_ENV_PREFIX = "DBT_ENV_SECRET"
+SECRET_ENV_PREFIX = "DBT_ENV_SECRET_"
 DEFAULT_ENV_PLACEHOLDER = "DBT_DEFAULT_PLACEHOLDER"
 METADATA_ENV_PREFIX = "DBT_ENV_CUSTOM_ENV_"
 
@@ -9,11 +9,8 @@ PIN_PACKAGE_URL = (
     "https://docs.getdbt.com/docs/package-management#section-specifying-package-versions"
 )
 
-DBT_PROJECT_FILE_NAME = "dbt_project.yml"
 PACKAGES_FILE_NAME = "packages.yml"
 DEPENDENCIES_FILE_NAME = "dependencies.yml"
-PACKAGE_LOCK_FILE_NAME = "package-lock.yml"
 MANIFEST_FILE_NAME = "manifest.json"
 SEMANTIC_MANIFEST_FILE_NAME = "semantic_manifest.json"
 PARTIAL_PARSE_FILE_NAME = "partial_parse.msgpack"
-PACKAGE_LOCK_HASH_KEY = "sha1_hash"

core/dbt/context/base.py

@@ -1,8 +1,6 @@
-from __future__ import annotations
-
 import json
 import os
-from typing import Any, Callable, Dict, NoReturn, Optional, Mapping, Iterable, Set, List
+from typing import Any, Dict, NoReturn, Optional, Mapping, Iterable, Set, List
 import threading
 
 from dbt.flags import get_flags
@@ -88,29 +86,33 @@ def get_context_modules() -> Dict[str, Dict[str, Any]]:
 
 
 class ContextMember:
-    def __init__(self, value: Any, name: Optional[str] = None) -> None:
+    def __init__(self, value, name=None):
         self.name = name
         self.inner = value
 
-    def key(self, default: str) -> str:
+    def key(self, default):
         if self.name is None:
             return default
         return self.name
 
 
-def contextmember(value: Optional[str] = None) -> Callable:
+def contextmember(value):
-    return lambda v: ContextMember(v, name=value)
+    if isinstance(value, str):
+        return lambda v: ContextMember(v, name=value)
+    return ContextMember(value)
 
 
-def contextproperty(value: Optional[str] = None) -> Callable:
+def contextproperty(value):
-    return lambda v: ContextMember(property(v), name=value)
+    if isinstance(value, str):
+        return lambda v: ContextMember(property(v), name=value)
+    return ContextMember(property(value))
 
 
 class ContextMeta(type):
-    def __new__(mcls, name, bases, dct: Dict[str, Any]) -> ContextMeta:
+    def __new__(mcls, name, bases, dct):
-        context_members: Dict[str, Any] = {}
+        context_members = {}
-        context_attrs: Dict[str, Any] = {}
+        context_attrs = {}
-        new_dct: Dict[str, Any] = {}
+        new_dct = {}
 
         for base in bases:
             context_members.update(getattr(base, "_context_members_", {}))
@@ -146,28 +148,27 @@ class Var:
         return self._cli_vars
 
     @property
-    def node_name(self) -> str:
+    def node_name(self):
         if self._node is not None:
             return self._node.name
         else:
             return "<Configuration>"
 
-    def get_missing_var(self, var_name: str) -> NoReturn:
+    def get_missing_var(self, var_name):
-        # TODO function name implies a non exception resolution
+        raise RequiredVarNotFoundError(var_name, self._merged, self._node)
-        raise RequiredVarNotFoundError(var_name, dict(self._merged), self._node)
 
-    def has_var(self, var_name: str) -> bool:
+    def has_var(self, var_name: str):
         return var_name in self._merged
 
-    def get_rendered_var(self, var_name: str) -> Any:
+    def get_rendered_var(self, var_name):
         raw = self._merged[var_name]
         # if bool/int/float/etc are passed in, don't compile anything
         if not isinstance(raw, str):
             return raw
 
-        return get_rendered(raw, dict(self._context))
+        return get_rendered(raw, self._context)
 
-    def __call__(self, var_name: str, default: Any = _VAR_NOTSET) -> Any:
+    def __call__(self, var_name, default=_VAR_NOTSET):
         if self.has_var(var_name):
             return self.get_rendered_var(var_name)
         elif default is not self._VAR_NOTSET:
@@ -177,17 +178,13 @@ class Var:
 
 
 class BaseContext(metaclass=ContextMeta):
-    # Set by ContextMeta
-    _context_members_: Dict[str, Any]
-    _context_attrs_: Dict[str, Any]
-
     # subclass is TargetContext
-    def __init__(self, cli_vars: Dict[str, Any]) -> None:
+    def __init__(self, cli_vars):
-        self._ctx: Dict[str, Any] = {}
+        self._ctx = {}
-        self.cli_vars: Dict[str, Any] = cli_vars
+        self.cli_vars = cli_vars
-        self.env_vars: Dict[str, Any] = {}
+        self.env_vars = {}
 
-    def generate_builtins(self) -> Dict[str, Any]:
+    def generate_builtins(self):
         builtins: Dict[str, Any] = {}
         for key, value in self._context_members_.items():
             if hasattr(value, "__get__"):
@@ -197,14 +194,14 @@ class BaseContext(metaclass=ContextMeta):
         return builtins
 
     # no dbtClassMixin so this is not an actual override
-    def to_dict(self) -> Dict[str, Any]:
+    def to_dict(self):
         self._ctx["context"] = self._ctx
         builtins = self.generate_builtins()
         self._ctx["builtins"] = builtins
         self._ctx.update(builtins)
         return self._ctx
 
-    @contextproperty()
+    @contextproperty
     def dbt_version(self) -> str:
         """The `dbt_version` variable returns the installed version of dbt that
         is currently running. It can be used for debugging or auditing
@@ -224,7 +221,7 @@ class BaseContext(metaclass=ContextMeta):
         """
         return dbt_version
 
-    @contextproperty()
+    @contextproperty
     def var(self) -> Var:
         """Variables can be passed from your `dbt_project.yml` file into models
         during compilation. These variables are useful for configuring packages
@@ -293,7 +290,7 @@ class BaseContext(metaclass=ContextMeta):
         """
         return Var(self._ctx, self.cli_vars)
 
-    @contextmember()
+    @contextmember
     def env_var(self, var: str, default: Optional[str] = None) -> str:
         """The env_var() function. Return the environment variable named 'var'.
         If there is no such environment variable set, return the default.
@@ -321,7 +318,7 @@ class BaseContext(metaclass=ContextMeta):
 
     if os.environ.get("DBT_MACRO_DEBUGGING"):
 
-        @contextmember()
+        @contextmember
         @staticmethod
         def debug():
             """Enter a debugger at this line in the compiled jinja code."""
@@ -360,7 +357,7 @@ class BaseContext(metaclass=ContextMeta):
         """
         raise MacroReturn(data)
 
-    @contextmember()
+    @contextmember
     @staticmethod
     def fromjson(string: str, default: Any = None) -> Any:
         """The `fromjson` context method can be used to deserialize a json
@@ -381,7 +378,7 @@ class BaseContext(metaclass=ContextMeta):
         except ValueError:
             return default
 
-    @contextmember()
+    @contextmember
     @staticmethod
     def tojson(value: Any, default: Any = None, sort_keys: bool = False) -> Any:
         """The `tojson` context method can be used to serialize a Python
@@ -404,7 +401,7 @@ class BaseContext(metaclass=ContextMeta):
         except ValueError:
             return default
 
-    @contextmember()
+    @contextmember
     @staticmethod
     def fromyaml(value: str, default: Any = None) -> Any:
         """The fromyaml context method can be used to deserialize a yaml string
@@ -435,7 +432,7 @@ class BaseContext(metaclass=ContextMeta):
 
     # safe_dump defaults to sort_keys=True, but we act like json.dumps (the
     # opposite)
-    @contextmember()
+    @contextmember
     @staticmethod
     def toyaml(
         value: Any, default: Optional[str] = None, sort_keys: bool = False
@@ -480,7 +477,7 @@ class BaseContext(metaclass=ContextMeta):
         except TypeError:
             return default
 
-    @contextmember()
+    @contextmember
     @staticmethod
     def set_strict(value: Iterable[Any]) -> Set[Any]:
         """The `set_strict` context method can be used to convert any iterable
@@ -522,7 +519,7 @@ class BaseContext(metaclass=ContextMeta):
         except TypeError:
             return default
 
-    @contextmember()
+    @contextmember
     @staticmethod
     def zip_strict(*args: Iterable[Any]) -> Iterable[Any]:
         """The `zip_strict` context method can be used to used to return
@@ -544,7 +541,7 @@ class BaseContext(metaclass=ContextMeta):
         except TypeError as e:
             raise ZipStrictWrongTypeError(e)
 
-    @contextmember()
+    @contextmember
     @staticmethod
     def log(msg: str, info: bool = False) -> str:
         """Logs a line to either the log file or stdout.
@@ -565,7 +562,7 @@ class BaseContext(metaclass=ContextMeta):
             fire_event(JinjaLogDebug(msg=msg, node_info=get_node_info()))
         return ""
 
-    @contextproperty()
+    @contextproperty
     def run_started_at(self) -> Optional[datetime.datetime]:
         """`run_started_at` outputs the timestamp that this run started, e.g.
         `2017-04-21 01:23:45.678`. The `run_started_at` variable is a Python
@@ -593,19 +590,19 @@ class BaseContext(metaclass=ContextMeta):
         else:
             return None
 
-    @contextproperty()
+    @contextproperty
     def invocation_id(self) -> Optional[str]:
         """invocation_id outputs a UUID generated for this dbt run (useful for
         auditing)
         """
         return get_invocation_id()
 
-    @contextproperty()
+    @contextproperty
     def thread_id(self) -> str:
         """thread_id outputs an ID for the current thread (useful for auditing)"""
         return threading.current_thread().name
 
-    @contextproperty()
+    @contextproperty
     def modules(self) -> Dict[str, Any]:
         """The `modules` variable in the Jinja context contains useful Python
         modules for operating on data.
@@ -630,7 +627,7 @@ class BaseContext(metaclass=ContextMeta):
         """  # noqa
         return get_context_modules()
 
-    @contextproperty()
+    @contextproperty
     def flags(self) -> Any:
         """The `flags` variable contains true/false values for flags provided
         on the command line.
@@ -647,7 +644,7 @@ class BaseContext(metaclass=ContextMeta):
         """
         return flags_module.get_flag_obj()
 
-    @contextmember()
+    @contextmember
     @staticmethod
     def print(msg: str) -> str:
         """Prints a line to stdout.
@@ -665,7 +662,7 @@ class BaseContext(metaclass=ContextMeta):
             print(msg)
         return ""
 
-    @contextmember()
+    @contextmember
     @staticmethod
     def diff_of_two_dicts(
         dict_a: Dict[str, List[str]], dict_b: Dict[str, List[str]]
@@ -694,7 +691,7 @@ class BaseContext(metaclass=ContextMeta):
                 dict_diff.update({k: dict_a[k]})
         return dict_diff
 
-    @contextmember()
+    @contextmember
     @staticmethod
     def local_md5(value: str) -> str:
         """Calculates an MD5 hash of the given string.

core/dbt/context/configured.py

@@ -19,7 +19,7 @@ class ConfiguredContext(TargetContext):
         super().__init__(config.to_target_dict(), config.cli_vars)
         self.config = config
 
-    @contextproperty()
+    @contextproperty
     def project_name(self) -> str:
         return self.config.project_name
 
@@ -80,11 +80,11 @@ class SchemaYamlContext(ConfiguredContext):
         self._project_name = project_name
         self.schema_yaml_vars = schema_yaml_vars
 
-    @contextproperty()
+    @contextproperty
     def var(self) -> ConfiguredVar:
         return ConfiguredVar(self._ctx, self.config, self._project_name)
 
-    @contextmember()
+    @contextmember
     def env_var(self, var: str, default: Optional[str] = None) -> str:
         return_value = None
         if var.startswith(SECRET_ENV_PREFIX):
@@ -113,7 +113,7 @@ class MacroResolvingContext(ConfiguredContext):
     def __init__(self, config):
         super().__init__(config)
 
-    @contextproperty()
+    @contextproperty
     def var(self) -> ConfiguredVar:
         return ConfiguredVar(self._ctx, self.config, self.config.project_name)
 

core/dbt/context/context_config.py

@@ -45,10 +45,6 @@ class UnrenderedConfig(ConfigSource):
             model_configs = unrendered.get("tests")
         elif resource_type == NodeType.Metric:
             model_configs = unrendered.get("metrics")
-        elif resource_type == NodeType.SemanticModel:
-            model_configs = unrendered.get("semantic_models")
-        elif resource_type == NodeType.SavedQuery:
-            model_configs = unrendered.get("saved_queries")
         elif resource_type == NodeType.Exposure:
             model_configs = unrendered.get("exposures")
         else:
@@ -74,10 +70,6 @@ class RenderedConfig(ConfigSource):
             model_configs = self.project.tests
         elif resource_type == NodeType.Metric:
             model_configs = self.project.metrics
-        elif resource_type == NodeType.SemanticModel:
-            model_configs = self.project.semantic_models
-        elif resource_type == NodeType.SavedQuery:
-            model_configs = self.project.saved_queries
         elif resource_type == NodeType.Exposure:
             model_configs = self.project.exposures
         else:
@@ -197,21 +189,9 @@ class ContextConfigGenerator(BaseContextConfigGenerator[C]):
 
     def _update_from_config(self, result: C, partial: Dict[str, Any], validate: bool = False) -> C:
         translated = self._active_project.credentials.translate_aliases(partial)
-        translated = self.translate_hook_names(translated)
+        return result.update_from(
-        updated = result.update_from(
             translated, self._active_project.credentials.type, validate=validate
         )
-        return updated
-
-    def translate_hook_names(self, project_dict):
-        # This is a kind of kludge because the fix for #6411 specifically allowed misspelling
-        # the hook field names in dbt_project.yml, which only ever worked because we didn't
-        # run validate on the dbt_project configs.
-        if "pre_hook" in project_dict:
-            project_dict["pre-hook"] = project_dict.pop("pre_hook")
-        if "post_hook" in project_dict:
-            project_dict["post-hook"] = project_dict.pop("post_hook")
-        return project_dict
 
     def calculate_node_config_dict(
         self,

core/dbt/context/docs.py

@@ -24,7 +24,7 @@ class DocsRuntimeContext(SchemaYamlContext):
         self.node = node
         self.manifest = manifest
 
-    @contextmember()
+    @contextmember
     def doc(self, *args: str) -> str:
         """The `doc` function is used to reference docs blocks in schema.yml
         files. It is analogous to the `ref` function. For more information,

core/dbt/context/exceptions_jinja.py

@@ -2,6 +2,7 @@ import functools
 from typing import NoReturn
 
 from dbt.events.functions import warn_or_error
+from dbt.events.helpers import env_secrets, scrub_secrets
 from dbt.events.types import JinjaLogWarning
 
 from dbt.exceptions import (
@@ -25,8 +26,6 @@ from dbt.exceptions import (
     ContractError,
     ColumnTypeMissingError,
     FailFastError,
-    scrub_secrets,
-    env_secrets,
 )
 
 

core/dbt/context/macro_resolver.py

@@ -40,7 +40,7 @@ class MacroResolver:
         self._build_internal_packages_namespace()
         self._build_macros_by_name()
 
-    def _build_internal_packages_namespace(self) -> None:
+    def _build_internal_packages_namespace(self):
         # Iterate in reverse-order and overwrite: the packages that are first
         # in the list are the ones we want to "win".
         self.internal_packages_namespace: MacroNamespace = {}
@@ -56,7 +56,7 @@ class MacroResolver:
     # root package namespace
     # non-internal packages (that aren't local or root)
     # dbt internal packages
-    def _build_macros_by_name(self) -> None:
+    def _build_macros_by_name(self):
         macros_by_name = {}
 
         # all internal packages (already in the right order)
@@ -78,7 +78,7 @@ class MacroResolver:
         self,
         package_namespaces: Dict[str, MacroNamespace],
         macro: Macro,
-    ) -> None:
+    ):
         if macro.package_name in package_namespaces:
             namespace = package_namespaces[macro.package_name]
         else:
@@ -89,7 +89,7 @@ class MacroResolver:
             raise DuplicateMacroNameError(macro, macro, macro.package_name)
         package_namespaces[macro.package_name][macro.name] = macro
 
-    def add_macro(self, macro: Macro) -> None:
+    def add_macro(self, macro: Macro):
         macro_name: str = macro.name
 
         # internal macros (from plugins) will be processed separately from
@@ -103,11 +103,11 @@ class MacroResolver:
             if macro.package_name == self.root_project_name:
                 self.root_package_macros[macro_name] = macro
 
-    def add_macros(self) -> None:
+    def add_macros(self):
         for macro in self.macros.values():
             self.add_macro(macro)
 
-    def get_macro(self, local_package, macro_name) -> Optional[Macro]:
+    def get_macro(self, local_package, macro_name):
         local_package_macros = {}
         # If the macro is explicitly prefixed with an internal namespace
         # (e.g. 'dbt.some_macro'), look there first
@@ -125,7 +125,7 @@ class MacroResolver:
             return self.macros_by_name[macro_name]
         return None
 
-    def get_macro_id(self, local_package, macro_name) -> Optional[str]:
+    def get_macro_id(self, local_package, macro_name):
         macro = self.get_macro(local_package, macro_name)
         if macro is None:
             return None

core/dbt/context/manifest.py

@@ -67,7 +67,7 @@ class ManifestContext(ConfiguredContext):
             dct.update(self.namespace)
         return dct
 
-    @contextproperty()
+    @contextproperty
     def context_macro_stack(self):
         return self.macro_stack
 

core/dbt/context/providers.py

@@ -60,6 +60,7 @@ from dbt.exceptions import (
     MetricArgsError,
     MissingConfigError,
     OperationsCannotRefEphemeralNodesError,
+    PackageNotInDepsError,
     ParsingError,
     RefBadContextError,
     RefArgsError,
@@ -617,7 +618,7 @@ class RuntimeMetricResolver(BaseMetricResolver):
                 target_package=target_package,
             )
 
-        return ResolvedMetricReference(target_metric, self.manifest)
+        return ResolvedMetricReference(target_metric, self.manifest, self.Relation)
 
 
 # `var` implementations.
@@ -637,8 +638,10 @@ class ModelConfiguredVar(Var):
         package_name = self._node.package_name
 
         if package_name != self._config.project_name:
-            if package_name in dependencies:
+            if package_name not in dependencies:
-                yield dependencies[package_name]
+                # I don't think this is actually reachable
+                raise PackageNotInDepsError(package_name, node=self._node)
+            yield dependencies[package_name]
         yield self._config
 
     def _generate_merged(self) -> Mapping[str, Any]:
@@ -751,19 +754,19 @@ class ProviderContext(ManifestContext):
             self.model,
         )
 
-    @contextproperty()
+    @contextproperty
     def dbt_metadata_envs(self) -> Dict[str, str]:
         return get_metadata_vars()
 
-    @contextproperty()
+    @contextproperty
     def invocation_args_dict(self):
         return args_to_dict(self.config.args)
 
-    @contextproperty()
+    @contextproperty
     def _sql_results(self) -> Dict[str, Optional[AttrDict]]:
         return self.sql_results
 
-    @contextmember()
+    @contextmember
     def load_result(self, name: str) -> Optional[AttrDict]:
         if name in self.sql_results:
             # handle the special case of "main" macro
@@ -784,7 +787,7 @@ class ProviderContext(ManifestContext):
             # Handle trying to load a result that was never stored
             return None
 
-    @contextmember()
+    @contextmember
     def store_result(
         self, name: str, response: Any, agate_table: Optional[agate.Table] = None
     ) -> str:
@@ -800,7 +803,7 @@ class ProviderContext(ManifestContext):
         )
         return ""
 
-    @contextmember()
+    @contextmember
     def store_raw_result(
         self,
         name: str,
@@ -812,7 +815,7 @@ class ProviderContext(ManifestContext):
         response = AdapterResponse(_message=message, code=code, rows_affected=rows_affected)
         return self.store_result(name, response, agate_table)
 
-    @contextproperty()
+    @contextproperty
     def validation(self):
         def validate_any(*args) -> Callable[[T], None]:
             def inner(value: T) -> None:
@@ -833,7 +836,7 @@ class ProviderContext(ManifestContext):
             }
         )
 
-    @contextmember()
+    @contextmember
     def write(self, payload: str) -> str:
         # macros/source defs aren't 'writeable'.
         if isinstance(self.model, (Macro, SourceDefinition)):
@@ -842,11 +845,11 @@ class ProviderContext(ManifestContext):
         self.model.write_node(self.config.project_root, self.model.build_path, payload)
         return ""
 
-    @contextmember()
+    @contextmember
     def render(self, string: str) -> str:
         return get_rendered(string, self._ctx, self.model)
 
-    @contextmember()
+    @contextmember
     def try_or_compiler_error(
         self, message_if_exception: str, func: Callable, *args, **kwargs
     ) -> Any:
@@ -855,32 +858,21 @@ class ProviderContext(ManifestContext):
         except Exception:
             raise CompilationError(message_if_exception, self.model)
 
-    @contextmember()
+    @contextmember
     def load_agate_table(self) -> agate.Table:
         if not isinstance(self.model, SeedNode):
             raise LoadAgateTableNotSeedError(self.model.resource_type, node=self.model)
-
+        assert self.model.root_path
-        # include package_path for seeds defined in packages
+        path = os.path.join(self.model.root_path, self.model.original_file_path)
-        package_path = (
-            os.path.join(self.config.packages_install_path, self.model.package_name)
-            if self.model.package_name != self.config.project_name
-            else "."
-        )
-        path = os.path.join(self.config.project_root, package_path, self.model.original_file_path)
-        if not os.path.exists(path):
-            assert self.model.root_path
-            path = os.path.join(self.model.root_path, self.model.original_file_path)
-
         column_types = self.model.config.column_types
-        delimiter = self.model.config.delimiter
         try:
-            table = agate_helper.from_csv(path, text_columns=column_types, delimiter=delimiter)
+            table = agate_helper.from_csv(path, text_columns=column_types)
         except ValueError as e:
             raise LoadAgateTableValueError(e, node=self.model)
         table.original_abspath = os.path.abspath(path)
         return table
 
-    @contextproperty()
+    @contextproperty
     def ref(self) -> Callable:
         """The most important function in dbt is `ref()`; it's impossible to
         build even moderately complex models without it. `ref()` is how you
@@ -921,11 +913,11 @@ class ProviderContext(ManifestContext):
         """
         return self.provider.ref(self.db_wrapper, self.model, self.config, self.manifest)
 
-    @contextproperty()
+    @contextproperty
     def source(self) -> Callable:
         return self.provider.source(self.db_wrapper, self.model, self.config, self.manifest)
 
-    @contextproperty()
+    @contextproperty
     def metric(self) -> Callable:
         return self.provider.metric(self.db_wrapper, self.model, self.config, self.manifest)
 
@@ -986,7 +978,7 @@ class ProviderContext(ManifestContext):
         """  # noqa
         return self.provider.Config(self.model, self.context_config)
 
-    @contextproperty()
+    @contextproperty
     def execute(self) -> bool:
         """`execute` is a Jinja variable that returns True when dbt is in
         "execute" mode.
@@ -1047,7 +1039,7 @@ class ProviderContext(ManifestContext):
         """  # noqa
         return self.provider.execute
 
-    @contextproperty()
+    @contextproperty
     def exceptions(self) -> Dict[str, Any]:
         """The exceptions namespace can be used to raise warnings and errors in
         dbt userspace.
@@ -1085,15 +1077,15 @@ class ProviderContext(ManifestContext):
         """  # noqa
         return wrapped_exports(self.model)
 
-    @contextproperty()
+    @contextproperty
     def database(self) -> str:
         return self.config.credentials.database
 
-    @contextproperty()
+    @contextproperty
     def schema(self) -> str:
         return self.config.credentials.schema
 
-    @contextproperty()
+    @contextproperty
     def var(self) -> ModelConfiguredVar:
         return self.provider.Var(
             context=self._ctx,
@@ -1110,22 +1102,22 @@ class ProviderContext(ManifestContext):
         """
         return self.db_wrapper
 
-    @contextproperty()
+    @contextproperty
     def api(self) -> Dict[str, Any]:
         return {
             "Relation": self.db_wrapper.Relation,
             "Column": self.adapter.Column,
         }
 
-    @contextproperty()
+    @contextproperty
     def column(self) -> Type[Column]:
         return self.adapter.Column
 
-    @contextproperty()
+    @contextproperty
     def env(self) -> Dict[str, Any]:
         return self.target
 
-    @contextproperty()
+    @contextproperty
     def graph(self) -> Dict[str, Any]:
         """The `graph` context variable contains information about the nodes in
         your dbt project. Models, sources, tests, and snapshots are all
@@ -1234,42 +1226,30 @@ class ProviderContext(ManifestContext):
 
     @contextproperty("model")
     def ctx_model(self) -> Dict[str, Any]:
-        model_dct = self.model.to_dict(omit_none=True)
+        ret = self.model.to_dict(omit_none=True)
         # Maintain direct use of compiled_sql
         # TODO add depreciation logic[CT-934]
-        if "compiled_code" in model_dct:
+        if "compiled_code" in ret:
-            model_dct["compiled_sql"] = model_dct["compiled_code"]
+            ret["compiled_sql"] = ret["compiled_code"]
+        return ret
 
-        if (
+    @contextproperty
-            hasattr(self.model, "contract")
-            and self.model.contract.alias_types is True
-            and "columns" in model_dct
-        ):
-            for column in model_dct["columns"].values():
-                if "data_type" in column:
-                    orig_data_type = column["data_type"]
-                    # translate data_type to value in Column.TYPE_LABELS
-                    new_data_type = self.adapter.Column.translate_type(orig_data_type)
-                    column["data_type"] = new_data_type
-        return model_dct
-
-    @contextproperty()
     def pre_hooks(self) -> Optional[List[Dict[str, Any]]]:
         return None
 
-    @contextproperty()
+    @contextproperty
     def post_hooks(self) -> Optional[List[Dict[str, Any]]]:
         return None
 
-    @contextproperty()
+    @contextproperty
     def sql(self) -> Optional[str]:
         return None
 
-    @contextproperty()
+    @contextproperty
     def sql_now(self) -> str:
         return self.adapter.date_function()
 
-    @contextmember()
+    @contextmember
     def adapter_macro(self, name: str, *args, **kwargs):
         """This was deprecated in v0.18 in favor of adapter.dispatch"""
         msg = (
@@ -1281,7 +1261,7 @@ class ProviderContext(ManifestContext):
         )
         raise CompilationError(msg)
 
-    @contextmember()
+    @contextmember
     def env_var(self, var: str, default: Optional[str] = None) -> str:
         """The env_var() function. Return the environment variable named 'var'.
         If there is no such environment variable set, return the default.
@@ -1325,7 +1305,7 @@ class ProviderContext(ManifestContext):
         else:
             raise EnvVarMissingError(var)
 
-    @contextproperty()
+    @contextproperty
     def selected_resources(self) -> List[str]:
         """The `selected_resources` variable contains a list of the resources
         selected based on the parameters provided to the dbt command.
@@ -1334,7 +1314,7 @@ class ProviderContext(ManifestContext):
         """
         return selected_resources.SELECTED_RESOURCES
 
-    @contextmember()
+    @contextmember
     def submit_python_job(self, parsed_model: Dict, compiled_code: str) -> AdapterResponse:
         # Check macro_stack and that the unique id is for a materialization macro
         if not (
@@ -1377,7 +1357,7 @@ class MacroContext(ProviderContext):
 class ModelContext(ProviderContext):
     model: ManifestNode
 
-    @contextproperty()
+    @contextproperty
     def pre_hooks(self) -> List[Dict[str, Any]]:
         if self.model.resource_type in [NodeType.Source, NodeType.Test]:
             return []
@@ -1386,7 +1366,7 @@ class ModelContext(ProviderContext):
             h.to_dict(omit_none=True) for h in self.model.config.pre_hook  # type: ignore[union-attr] # noqa
         ]
 
-    @contextproperty()
+    @contextproperty
     def post_hooks(self) -> List[Dict[str, Any]]:
         if self.model.resource_type in [NodeType.Source, NodeType.Test]:
             return []
@@ -1395,7 +1375,7 @@ class ModelContext(ProviderContext):
             h.to_dict(omit_none=True) for h in self.model.config.post_hook  # type: ignore[union-attr] # noqa
         ]
 
-    @contextproperty()
+    @contextproperty
     def sql(self) -> Optional[str]:
         # only doing this in sql model for backward compatible
         if self.model.language == ModelLanguage.sql:  # type: ignore[union-attr]
@@ -1412,7 +1392,7 @@ class ModelContext(ProviderContext):
         else:
             return None
 
-    @contextproperty()
+    @contextproperty
     def compiled_code(self) -> Optional[str]:
         if getattr(self.model, "defer_relation", None):
             # TODO https://github.com/dbt-labs/dbt-core/issues/7976
@@ -1423,15 +1403,15 @@ class ModelContext(ProviderContext):
         else:
             return None
 
-    @contextproperty()
+    @contextproperty
     def database(self) -> str:
         return getattr(self.model, "database", self.config.credentials.database)
 
-    @contextproperty()
+    @contextproperty
     def schema(self) -> str:
         return getattr(self.model, "schema", self.config.credentials.schema)
 
-    @contextproperty()
+    @contextproperty
     def this(self) -> Optional[RelationProxy]:
         """`this` makes available schema information about the currently
         executing model. It's is useful in any context in which you need to
@@ -1466,7 +1446,7 @@ class ModelContext(ProviderContext):
             return None
         return self.db_wrapper.Relation.create_from(self.config, self.model)
 
-    @contextproperty()
+    @contextproperty
     def defer_relation(self) -> Optional[RelationProxy]:
         """
         For commands which add information about this node's corresponding
@@ -1680,7 +1660,7 @@ class TestContext(ProviderContext):
         )
         self.namespace = macro_namespace
 
-    @contextmember()
+    @contextmember
     def env_var(self, var: str, default: Optional[str] = None) -> str:
         return_value = None
         if var.startswith(SECRET_ENV_PREFIX):

core/dbt/context/secret.py

@@ -14,14 +14,14 @@ class SecretContext(BaseContext):
     """This context is used in profiles.yml + packages.yml. It can render secret
     env vars that aren't usable elsewhere"""
 
-    @contextmember()
+    @contextmember
     def env_var(self, var: str, default: Optional[str] = None) -> str:
         """The env_var() function. Return the environment variable named 'var'.
         If there is no such environment variable set, return the default.
 
         If the default is None, raise an exception for an undefined variable.
 
-        In this context *only*, env_var will accept env vars prefixed with DBT_ENV_SECRET.
+        In this context *only*, env_var will accept env vars prefixed with DBT_ENV_SECRET_.
         It will return the name of the secret env var, wrapped in 'start' and 'end' identifiers.
         The actual value will be subbed in later in SecretRenderer.render_value()
         """

core/dbt/context/target.py

@@ -9,7 +9,7 @@ class TargetContext(BaseContext):
         super().__init__(cli_vars=cli_vars)
         self.target_dict = target_dict
 
-    @contextproperty()
+    @contextproperty
     def target(self) -> Dict[str, Any]:
         """`target` contains information about your connection to the warehouse
         (specified in profiles.yml). Some configs are shared between all

core/dbt/contracts/connection.py

@@ -16,21 +16,26 @@ from dbt.utils import translate_aliases, md5
 from dbt.events.functions import fire_event
 from dbt.events.types import NewConnectionOpening
 from dbt.events.contextvars import get_node_info
-from typing_extensions import Protocol, Annotated
+from typing_extensions import Protocol
 from dbt.dataclass_schema import (
     dbtClassMixin,
     StrEnum,
     ExtensibleDbtClassMixin,
+    HyphenatedDbtClassMixin,
     ValidatedStringMixin,
+    register_pattern,
 )
 from dbt.contracts.util import Replaceable
-from mashumaro.jsonschema.annotations import Pattern
 
 
 class Identifier(ValidatedStringMixin):
     ValidationRegex = r"^[A-Za-z_][A-Za-z0-9_]+$"
 
 
+# we need register_pattern for jsonschema validation
+register_pattern(Identifier, r"^[A-Za-z_][A-Za-z0-9_]+$")
+
+
 @dataclass
 class AdapterResponse(dbtClassMixin):
     _message: str
@@ -50,8 +55,7 @@ class ConnectionState(StrEnum):
 
 @dataclass(init=False)
 class Connection(ExtensibleDbtClassMixin, Replaceable):
-    # Annotated is used by mashumaro for jsonschema generation
+    type: Identifier
-    type: Annotated[Identifier, Pattern(r"^[A-Za-z_][A-Za-z0-9_]+$")]
     name: Optional[str] = None
     state: ConnectionState = ConnectionState.INIT
     transaction_open: bool = False
@@ -104,7 +108,7 @@ class LazyHandle:
     connection, updating the handle on the Connection.
     """
 
-    def __init__(self, opener: Callable[[Connection], Connection]) -> None:
+    def __init__(self, opener: Callable[[Connection], Connection]):
         self.opener = opener
 
     def resolve(self, connection: Connection) -> Connection:
@@ -157,7 +161,6 @@ class Credentials(ExtensibleDbtClassMixin, Replaceable, metaclass=abc.ABCMeta):
     @classmethod
     def __pre_deserialize__(cls, data):
         data = super().__pre_deserialize__(data)
-        # Need to fixup dbname => database, pass => password
         data = cls.translate_aliases(data)
         return data
 
@@ -178,9 +181,17 @@ class Credentials(ExtensibleDbtClassMixin, Replaceable, metaclass=abc.ABCMeta):
         return dct
 
 
+class UserConfigContract(Protocol):
+    send_anonymous_usage_stats: bool
+    use_colors: Optional[bool] = None
+    partial_parse: Optional[bool] = None
+    printer_width: Optional[int] = None
+
+
 class HasCredentials(Protocol):
     credentials: Credentials
     profile_name: str
+    user_config: UserConfigContract
     target_name: str
     threads: int
 
@@ -209,10 +220,10 @@ DEFAULT_QUERY_COMMENT = """
 
 
 @dataclass
-class QueryComment(dbtClassMixin):
+class QueryComment(HyphenatedDbtClassMixin):
     comment: str = DEFAULT_QUERY_COMMENT
     append: bool = False
-    job_label: bool = field(default=False, metadata={"alias": "job-label"})
+    job_label: bool = False
 
 
 class AdapterRequiredConfig(HasCredentials, Protocol):

core/dbt/contracts/files.py

@@ -225,13 +225,10 @@ class SchemaSourceFile(BaseSourceFile):
     sources: List[str] = field(default_factory=list)
     exposures: List[str] = field(default_factory=list)
     metrics: List[str] = field(default_factory=list)
-    # metrics generated from semantic_model measures
-    generated_metrics: List[str] = field(default_factory=list)
     groups: List[str] = field(default_factory=list)
     # node patches contain models, seeds, snapshots, analyses
     ndp: List[str] = field(default_factory=list)
     semantic_models: List[str] = field(default_factory=list)
-    saved_queries: List[str] = field(default_factory=list)
     # any macro patches in this file by macro unique_id.
     mcp: Dict[str, str] = field(default_factory=dict)
     # any source patches in this file. The entries are package, name pairs

core/dbt/contracts/graph/manifest.py

@@ -20,12 +20,10 @@ from typing import (
     Generic,
     AbstractSet,
     ClassVar,
-    Iterable,
 )
 from typing_extensions import Protocol
 from uuid import UUID
 
-
 from dbt.contracts.graph.nodes import (
     BaseNode,
     Documentation,
@@ -39,7 +37,6 @@ from dbt.contracts.graph.nodes import (
     ModelNode,
     DeferRelation,
     ResultNode,
-    SavedQuery,
     SemanticModel,
     SourceDefinition,
     UnpatchedSourceDefinition,
@@ -47,13 +44,7 @@ from dbt.contracts.graph.nodes import (
 from dbt.contracts.graph.unparsed import SourcePatch, NodeVersion, UnparsedVersion
 from dbt.contracts.graph.manifest_upgrade import upgrade_manifest_json
 from dbt.contracts.files import SourceFile, SchemaSourceFile, FileHash, AnySourceFile
-from dbt.contracts.util import (
+from dbt.contracts.util import BaseArtifactMetadata, SourceKey, ArtifactMixin, schema_version
-    BaseArtifactMetadata,
-    SourceKey,
-    ArtifactMixin,
-    schema_version,
-    get_artifact_schema_version,
-)
 from dbt.dataclass_schema import dbtClassMixin
 from dbt.exceptions import (
     CompilationError,
@@ -68,7 +59,7 @@ from dbt.events.types import MergedFromState, UnpinnedRefNewVersionAvailable
 from dbt.events.contextvars import get_node_info
 from dbt.node_types import NodeType, AccessType
 from dbt.flags import get_flags, MP_CONTEXT
-from dbt import tracking, deprecations
+from dbt import tracking
 import dbt.utils
 
 
@@ -97,7 +88,7 @@ def find_unique_id_for_package(storage, key, package: Optional[PackageName]):
 
 
 class DocLookup(dbtClassMixin):
-    def __init__(self, manifest: "Manifest") -> None:
+    def __init__(self, manifest: "Manifest"):
         self.storage: Dict[str, Dict[PackageName, UniqueID]] = {}
         self.populate(manifest)
 
@@ -128,7 +119,7 @@ class DocLookup(dbtClassMixin):
 
 
 class SourceLookup(dbtClassMixin):
-    def __init__(self, manifest: "Manifest") -> None:
+    def __init__(self, manifest: "Manifest"):
         self.storage: Dict[str, Dict[PackageName, UniqueID]] = {}
         self.populate(manifest)
 
@@ -165,7 +156,7 @@ class RefableLookup(dbtClassMixin):
     _lookup_types: ClassVar[set] = set(NodeType.refable())
     _versioned_types: ClassVar[set] = set(NodeType.versioned())
 
-    def __init__(self, manifest: "Manifest") -> None:
+    def __init__(self, manifest: "Manifest"):
         self.storage: Dict[str, Dict[PackageName, UniqueID]] = {}
         self.populate(manifest)
 
@@ -276,7 +267,7 @@ class RefableLookup(dbtClassMixin):
 
 
 class MetricLookup(dbtClassMixin):
-    def __init__(self, manifest: "Manifest") -> None:
+    def __init__(self, manifest: "Manifest"):
         self.storage: Dict[str, Dict[PackageName, UniqueID]] = {}
         self.populate(manifest)
 
@@ -308,41 +299,6 @@ class MetricLookup(dbtClassMixin):
         return manifest.metrics[unique_id]
 
 
-class SavedQueryLookup(dbtClassMixin):
-    """Lookup utility for finding SavedQuery nodes"""
-
-    def __init__(self, manifest: "Manifest") -> None:
-        self.storage: Dict[str, Dict[PackageName, UniqueID]] = {}
-        self.populate(manifest)
-
-    def get_unique_id(self, search_name, package: Optional[PackageName]):
-        return find_unique_id_for_package(self.storage, search_name, package)
-
-    def find(self, search_name, package: Optional[PackageName], manifest: "Manifest"):
-        unique_id = self.get_unique_id(search_name, package)
-        if unique_id is not None:
-            return self.perform_lookup(unique_id, manifest)
-        return None
-
-    def add_saved_query(self, saved_query: SavedQuery):
-        if saved_query.search_name not in self.storage:
-            self.storage[saved_query.search_name] = {}
-
-        self.storage[saved_query.search_name][saved_query.package_name] = saved_query.unique_id
-
-    def populate(self, manifest):
-        for saved_query in manifest.saved_queries.values():
-            if hasattr(saved_query, "name"):
-                self.add_saved_query(saved_query)
-
-    def perform_lookup(self, unique_id: UniqueID, manifest: "Manifest") -> SavedQuery:
-        if unique_id not in manifest.saved_queries:
-            raise dbt.exceptions.DbtInternalError(
-                f"SavedQUery {unique_id} found in cache but not found in manifest"
-            )
-        return manifest.saved_queries[unique_id]
-
-
 class SemanticModelByMeasureLookup(dbtClassMixin):
     """Lookup utility for finding SemanticModel by measure
 
@@ -350,7 +306,7 @@ class SemanticModelByMeasureLookup(dbtClassMixin):
     the semantic models in a manifest.
     """
 
-    def __init__(self, manifest: "Manifest") -> None:
+    def __init__(self, manifest: "Manifest"):
         self.storage: DefaultDict[str, Dict[PackageName, UniqueID]] = defaultdict(dict)
         self.populate(manifest)
 
@@ -375,31 +331,20 @@ class SemanticModelByMeasureLookup(dbtClassMixin):
         """Populate storage with all the measure + package paths to the Manifest's SemanticModels"""
         for semantic_model in manifest.semantic_models.values():
             self.add(semantic_model=semantic_model)
-        for disabled in manifest.disabled.values():
-            for node in disabled:
-                if isinstance(node, SemanticModel):
-                    self.add(semantic_model=node)
 
     def perform_lookup(self, unique_id: UniqueID, manifest: "Manifest") -> SemanticModel:
         """Tries to get a SemanticModel from the Manifest"""
-        enabled_semantic_model: Optional[SemanticModel] = manifest.semantic_models.get(unique_id)
+        semantic_model = manifest.semantic_models.get(unique_id)
-        disabled_semantic_model: Optional[List] = manifest.disabled.get(unique_id)
+        if semantic_model is None:
-
-        if isinstance(enabled_semantic_model, SemanticModel):
-            return enabled_semantic_model
-        elif disabled_semantic_model is not None and isinstance(
-            disabled_semantic_model[0], SemanticModel
-        ):
-            return disabled_semantic_model[0]
-        else:
             raise dbt.exceptions.DbtInternalError(
                 f"Semantic model `{unique_id}` found in cache but not found in manifest"
             )
+        return semantic_model
 
 
-# This handles both models/seeds/snapshots and sources/metrics/exposures/semantic_models
+# This handles both models/seeds/snapshots and sources/metrics/exposures
 class DisabledLookup(dbtClassMixin):
-    def __init__(self, manifest: "Manifest") -> None:
+    def __init__(self, manifest: "Manifest"):
         self.storage: Dict[str, Dict[PackageName, List[Any]]] = {}
         self.populate(manifest)
 
@@ -617,29 +562,11 @@ M = TypeVar("M", bound=MacroCandidate)
 
 
 class CandidateList(List[M]):
-    def last_candidate(
+    def last(self) -> Optional[Macro]:
-        self, valid_localities: Optional[List[Locality]] = None
-    ) -> Optional[MacroCandidate]:
-        """
-        Obtain the last (highest precedence) MacroCandidate from the CandidateList of any locality in valid_localities.
-        If valid_localities is not specified, return the last MacroCandidate of any locality.
-        """
         if not self:
             return None
         self.sort()
-
+        return self[-1].macro
-        if valid_localities is None:
-            return self[-1]
-
-        for candidate in reversed(self):
-            if candidate.locality in valid_localities:
-                return candidate
-
-        return None
-
-    def last(self) -> Optional[Macro]:
-        last_candidate = self.last_candidate()
-        return last_candidate.macro if last_candidate is not None else None
 
 
 def _get_locality(macro: Macro, root_project_name: str, internal_packages: Set[str]) -> Locality:
@@ -671,9 +598,6 @@ class Disabled(Generic[D]):
 MaybeMetricNode = Optional[Union[Metric, Disabled[Metric]]]
 
 
-MaybeSavedQueryNode = Optional[Union[SavedQuery, Disabled[SavedQuery]]]
-
-
 MaybeDocumentation = Optional[Documentation]
 
 
@@ -818,7 +742,6 @@ class Manifest(MacroMethods, DataClassMessagePackMixin, dbtClassMixin):
     disabled: MutableMapping[str, List[GraphMemberNode]] = field(default_factory=dict)
     env_vars: MutableMapping[str, str] = field(default_factory=dict)
     semantic_models: MutableMapping[str, SemanticModel] = field(default_factory=dict)
-    saved_queries: MutableMapping[str, SavedQuery] = field(default_factory=dict)
 
     _doc_lookup: Optional[DocLookup] = field(
         default=None, metadata={"serialize": lambda x: None, "deserialize": lambda x: None}
@@ -832,9 +755,6 @@ class Manifest(MacroMethods, DataClassMessagePackMixin, dbtClassMixin):
     _metric_lookup: Optional[MetricLookup] = field(
         default=None, metadata={"serialize": lambda x: None, "deserialize": lambda x: None}
     )
-    _saved_query_lookup: Optional[SavedQueryLookup] = field(
-        default=None, metadata={"serialize": lambda x: None, "deserialize": lambda x: None}
-    )
     _semantic_model_by_measure_lookup: Optional[SemanticModelByMeasureLookup] = field(
         default=None, metadata={"serialize": lambda x: None, "deserialize": lambda x: None}
     )
@@ -879,9 +799,6 @@ class Manifest(MacroMethods, DataClassMessagePackMixin, dbtClassMixin):
             "semantic_models": {
                 k: v.to_dict(omit_none=False) for k, v in self.semantic_models.items()
             },
-            "saved_queries": {
-                k: v.to_dict(omit_none=False) for k, v in self.saved_queries.items()
-            },
         }
 
     def build_disabled_by_file_id(self):
@@ -933,33 +850,7 @@ class Manifest(MacroMethods, DataClassMessagePackMixin, dbtClassMixin):
                 for specificity, atype in enumerate(self._get_parent_adapter_types(adapter_type))
             )
         )
-        core_candidates = [
+        return candidates.last()
-            candidate for candidate in candidates if candidate.locality == Locality.Core
-        ]
-
-        materialization_candidate = candidates.last_candidate()
-        # If an imported materialization macro was found that also had a core candidate, fire a deprecation
-        if (
-            materialization_candidate is not None
-            and materialization_candidate.locality == Locality.Imported
-            and core_candidates
-        ):
-            # preserve legacy behaviour - allow materialization override
-            if (
-                get_flags().require_explicit_package_overrides_for_builtin_materializations
-                is False
-            ):
-                deprecations.warn(
-                    "package-materialization-override",
-                    package_name=materialization_candidate.macro.package_name,
-                    materialization_name=materialization_name,
-                )
-            else:
-                materialization_candidate = candidates.last_candidate(
-                    valid_localities=[Locality.Core, Locality.Root]
-                )
-
-        return materialization_candidate.macro if materialization_candidate else None
 
     def get_resource_fqns(self) -> Mapping[str, PathSet]:
         resource_fqns: Dict[str, Set[Tuple[str, ...]]] = {}
@@ -969,7 +860,6 @@ class Manifest(MacroMethods, DataClassMessagePackMixin, dbtClassMixin):
             self.sources.values(),
             self.metrics.values(),
             self.semantic_models.values(),
-            self.saved_queries.values(),
         )
         for resource in all_resources:
             resource_type_plural = resource.resource_type.pluralize()
@@ -1005,7 +895,6 @@ class Manifest(MacroMethods, DataClassMessagePackMixin, dbtClassMixin):
             files={k: _deepcopy(v) for k, v in self.files.items()},
             state_check=_deepcopy(self.state_check),
             semantic_models={k: _deepcopy(v) for k, v in self.semantic_models.items()},
-            saved_queries={k: _deepcopy(v) for k, v in self.saved_queries.items()},
         )
         copy.build_flat_graph()
         return copy
@@ -1018,7 +907,6 @@ class Manifest(MacroMethods, DataClassMessagePackMixin, dbtClassMixin):
                 self.exposures.values(),
                 self.metrics.values(),
                 self.semantic_models.values(),
-                self.saved_queries.values(),
             )
         )
         forward_edges, backward_edges = build_node_edges(edge_members)
@@ -1039,22 +927,13 @@ class Manifest(MacroMethods, DataClassMessagePackMixin, dbtClassMixin):
         groupable_nodes = list(
             chain(
                 self.nodes.values(),
-                self.saved_queries.values(),
-                self.semantic_models.values(),
                 self.metrics.values(),
             )
         )
         group_map = {group.name: [] for group in self.groups.values()}
         for node in groupable_nodes:
             if node.group is not None:
-                # group updates are not included with state:modified and
+                group_map[node.group].append(node.unique_id)
-                # by ignoring the groups that aren't in the group map we
-                # can avoid hitting errors for groups that are not getting
-                # updated.  This is a hack but any groups that are not
-                # valid will be caught in
-                # parser.manifest.ManifestLoader.check_valid_group_config_node
-                if node.group in group_map:
-                    group_map[node.group].append(node.unique_id)
         self.group_map = group_map
 
     def writable_manifest(self) -> "WritableManifest":
@@ -1075,7 +954,6 @@ class Manifest(MacroMethods, DataClassMessagePackMixin, dbtClassMixin):
             parent_map=self.parent_map,
             group_map=self.group_map,
             semantic_models=self.semantic_models,
-            saved_queries=self.saved_queries,
         )
 
     def write(self, path):
@@ -1094,8 +972,6 @@ class Manifest(MacroMethods, DataClassMessagePackMixin, dbtClassMixin):
             return self.metrics[unique_id]
         elif unique_id in self.semantic_models:
             return self.semantic_models[unique_id]
-        elif unique_id in self.saved_queries:
-            return self.saved_queries[unique_id]
         else:
             # something terrible has happened
             raise dbt.exceptions.DbtInternalError(
@@ -1132,13 +1008,6 @@ class Manifest(MacroMethods, DataClassMessagePackMixin, dbtClassMixin):
             self._metric_lookup = MetricLookup(self)
         return self._metric_lookup
 
-    @property
-    def saved_query_lookup(self) -> SavedQueryLookup:
-        """Retuns a SavedQueryLookup, instantiating it first if necessary."""
-        if self._saved_query_lookup is None:
-            self._saved_query_lookup = SavedQueryLookup(self)
-        return self._saved_query_lookup
-
     @property
     def semantic_model_by_measure_lookup(self) -> SemanticModelByMeasureLookup:
         """Gets (and creates if necessary) the lookup utility for getting SemanticModels by measures"""
@@ -1187,7 +1056,8 @@ class Manifest(MacroMethods, DataClassMessagePackMixin, dbtClassMixin):
 
         return resolved_refs
 
-    # Called by dbt.parser.manifest._process_refs & ManifestLoader.check_for_model_deprecations
+    # Called by dbt.parser.manifest._process_refs_for_exposure, _process_refs_for_metric,
+    # and dbt.parser.manifest._process_refs_for_node
     def resolve_ref(
         self,
         source_node: GraphMemberNode,
@@ -1272,35 +1142,6 @@ class Manifest(MacroMethods, DataClassMessagePackMixin, dbtClassMixin):
             return Disabled(disabled[0])
         return None
 
-    def resolve_saved_query(
-        self,
-        target_saved_query_name: str,
-        target_saved_query_package: Optional[str],
-        current_project: str,
-        node_package: str,
-    ) -> MaybeSavedQueryNode:
-        """Tries to find the SavedQuery by name within the available project and packages.
-
-        Will return the first enabled SavedQuery matching the name found while iterating over
-        the scoped packages. If no enabled SavedQuery node match is found, returns the last
-        disabled SavedQuery node. Otherwise it returns None.
-        """
-        disabled: Optional[List[SavedQuery]] = None
-        candidates = _packages_to_search(current_project, node_package, target_saved_query_package)
-        for pkg in candidates:
-            saved_query = self.saved_query_lookup.find(target_saved_query_name, pkg, self)
-
-            if saved_query is not None and saved_query.config.enabled:
-                return saved_query
-
-            # it's possible that the node is disabled
-            if disabled is None:
-                disabled = self.disabled_lookup.find(f"{target_saved_query_name}", pkg)
-        if disabled:
-            return Disabled(disabled[0])
-
-        return None
-
     def resolve_semantic_model_for_measure(
         self,
         target_measure_name: str,
@@ -1315,7 +1156,6 @@ class Manifest(MacroMethods, DataClassMessagePackMixin, dbtClassMixin):
             semantic_model = self.semantic_model_by_measure_lookup.find(
                 target_measure_name, pkg, self
             )
-            # need to return it even if it's disabled so know it's not fully missing
             if semantic_model is not None:
                 return semantic_model
 
@@ -1491,13 +1331,10 @@ class Manifest(MacroMethods, DataClassMessagePackMixin, dbtClassMixin):
         self.exposures[exposure.unique_id] = exposure
         source_file.exposures.append(exposure.unique_id)
 
-    def add_metric(self, source_file: SchemaSourceFile, metric: Metric, generated: bool = False):
+    def add_metric(self, source_file: SchemaSourceFile, metric: Metric):
         _check_duplicates(metric, self.metrics)
         self.metrics[metric.unique_id] = metric
-        if not generated:
+        source_file.metrics.append(metric.unique_id)
-            source_file.metrics.append(metric.unique_id)
-        else:
-            source_file.generated_metrics.append(metric.unique_id)
 
     def add_group(self, source_file: SchemaSourceFile, group: Group):
         _check_duplicates(group, self.groups)
@@ -1519,10 +1356,6 @@ class Manifest(MacroMethods, DataClassMessagePackMixin, dbtClassMixin):
                 source_file.add_test(node.unique_id, test_from)
             if isinstance(node, Metric):
                 source_file.metrics.append(node.unique_id)
-            if isinstance(node, SavedQuery):
-                source_file.saved_queries.append(node.unique_id)
-            if isinstance(node, SemanticModel):
-                source_file.semantic_models.append(node.unique_id)
             if isinstance(node, Exposure):
                 source_file.exposures.append(node.unique_id)
         else:
@@ -1538,11 +1371,6 @@ class Manifest(MacroMethods, DataClassMessagePackMixin, dbtClassMixin):
         self.semantic_models[semantic_model.unique_id] = semantic_model
         source_file.semantic_models.append(semantic_model.unique_id)
 
-    def add_saved_query(self, source_file: SchemaSourceFile, saved_query: SavedQuery) -> None:
-        _check_duplicates(saved_query, self.saved_queries)
-        self.saved_queries[saved_query.unique_id] = saved_query
-        source_file.saved_queries.append(saved_query.unique_id)
-
     # end of methods formerly in ParseResult
 
     # Provide support for copy.deepcopy() - we just need to avoid the lock!
@@ -1570,7 +1398,6 @@ class Manifest(MacroMethods, DataClassMessagePackMixin, dbtClassMixin):
             self.disabled,
             self.env_vars,
             self.semantic_models,
-            self.saved_queries,
             self._doc_lookup,
             self._source_lookup,
             self._ref_lookup,
@@ -1583,19 +1410,19 @@ class Manifest(MacroMethods, DataClassMessagePackMixin, dbtClassMixin):
 
 
 class MacroManifest(MacroMethods):
-    def __init__(self, macros) -> None:
+    def __init__(self, macros):
         self.macros = macros
         self.metadata = ManifestMetadata()
         # This is returned by the 'graph' context property
         # in the ProviderContext class.
-        self.flat_graph: Dict[str, Any] = {}
+        self.flat_graph = {}
 
 
 AnyManifest = Union[Manifest, MacroManifest]
 
 
 @dataclass
-@schema_version("manifest", 11)
+@schema_version("manifest", 10)
 class WritableManifest(ArtifactMixin):
     nodes: Mapping[UniqueID, ManifestNode] = field(
         metadata=dict(description=("The nodes defined in the dbt project and its dependencies"))
@@ -1641,9 +1468,6 @@ class WritableManifest(ArtifactMixin):
             description="A mapping from group names to their nodes",
         )
     )
-    saved_queries: Mapping[UniqueID, SavedQuery] = field(
-        metadata=dict(description=("The saved queries defined in the dbt project"))
-    )
     semantic_models: Mapping[UniqueID, SemanticModel] = field(
         metadata=dict(description=("The semantic models defined in the dbt project"))
     )
@@ -1654,7 +1478,7 @@ class WritableManifest(ArtifactMixin):
     )
 
     @classmethod
-    def compatible_previous_versions(cls) -> Iterable[Tuple[str, int]]:
+    def compatible_previous_versions(self):
         return [
             ("manifest", 4),
             ("manifest", 5),
@@ -1662,15 +1486,14 @@ class WritableManifest(ArtifactMixin):
             ("manifest", 7),
             ("manifest", 8),
             ("manifest", 9),
-            ("manifest", 10),
         ]
 
     @classmethod
     def upgrade_schema_version(cls, data):
         """This overrides the "upgrade_schema_version" call in VersionedSchema (via
         ArtifactMixin) to modify the dictionary passed in from earlier versions of the manifest."""
-        manifest_schema_version = get_artifact_schema_version(data)
+        manifest_schema_version = get_manifest_schema_version(data)
-        if manifest_schema_version <= 10:
+        if manifest_schema_version <= 9:
             data = upgrade_manifest_json(data, manifest_schema_version)
         return cls.from_dict(data)
 
@@ -1683,6 +1506,13 @@ class WritableManifest(ArtifactMixin):
         return dct
 
 
+def get_manifest_schema_version(dct: dict) -> int:
+    schema_version = dct.get("metadata", {}).get("dbt_schema_version", None)
+    if not schema_version:
+        raise ValueError("Manifest doesn't have schema version")
+    return int(schema_version.split(".")[-2][-1])
+
+
 def _check_duplicates(value: BaseNode, src: Mapping[str, BaseNode]):
     if value.unique_id in src:
         raise DuplicateResourceNameError(value, src[value.unique_id])

core/dbt/contracts/graph/manifest_upgrade.py

@@ -62,72 +62,10 @@ def drop_v9_and_prior_metrics(manifest: dict) -> None:
     manifest["disabled"] = filtered_disabled_entries
 
 
-def _convert_dct_with_filter(v10_dct_with_opt_filter):
-    """Upgrage the filter object from v10 to v11.
-
-    v10 filters from a serialized manifest looked like:
-    {..., 'filter': {'where_sql_template': '<filter_value>'}}
-    whereas v11 filters look like:
-    {..., 'filter': {'where_filters': [{'where_sql_template': '<filter_value>'}, ...]}}
-    """
-    if v10_dct_with_opt_filter is not None and v10_dct_with_opt_filter.get("filter") is not None:
-        v10_dct_with_opt_filter["filter"] = {"where_filters": [v10_dct_with_opt_filter["filter"]]}
-
-
-def _convert_metric(v10_metric_dict):
-    """Upgrades a v10 metric object to a v11 metric object.
-
-    Specifcally the following properties change
-    1. metric.filter
-    2. metric.type_params.measure.filter
-    3. metric.type_params.input_measures[x].filter
-    4. metric.type_params.numerator.filter
-    5. metric.type_params.denominator.filter
-    6. metric.type_params.metrics[x].filter"
-    """
-
-    # handles top level metric filter
-    _convert_dct_with_filter(v10_metric_dict)
-
-    type_params = v10_metric_dict.get("type_params")
-    if type_params is not None:
-        _convert_dct_with_filter(type_params.get("measure"))
-        _convert_dct_with_filter(type_params.get("numerator"))
-        _convert_dct_with_filter(type_params.get("denominator"))
-
-        # handles metric.type_params.input_measures[x].filter
-        input_measures = type_params.get("input_measures")
-        if input_measures is not None:
-            for input_measure in input_measures:
-                _convert_dct_with_filter(input_measure)
-
-        # handles metric.type_params.metrics[x].filter
-        metrics = type_params.get("metrics")
-        if metrics is not None:
-            for metric in metrics:
-                _convert_dct_with_filter(metric)
-
-
-def upgrade_v10_metric_filters(manifest: dict):
-    """Handles metric filters changes from v10 to v11."""
-
-    metrics = manifest.get("metrics", {})
-    for metric in metrics.values():
-        _convert_metric(metric)
-
-    disabled_nodes = manifest.get("disabled", {})
-    for unique_id, nodes in disabled_nodes.items():
-        if unique_id.split(".")[0] == "metric":
-            for node in nodes:
-                _convert_metric(node)
-
-
 def upgrade_manifest_json(manifest: dict, manifest_schema_version: int) -> dict:
     # this should remain 9 while the check in `upgrade_schema_version` may change
     if manifest_schema_version <= 9:
         drop_v9_and_prior_metrics(manifest=manifest)
-    elif manifest_schema_version == 10:
-        upgrade_v10_metric_filters(manifest=manifest)
 
     for node_content in manifest.get("nodes", {}).values():
         upgrade_node_content(node_content)
@@ -166,6 +104,4 @@ def upgrade_manifest_json(manifest: dict, manifest_schema_version: int) -> dict:
         doc_content["resource_type"] = "doc"
     if "semantic_models" not in manifest:
         manifest["semantic_models"] = {}
-    if "saved_queries" not in manifest:
-        manifest["saved_queries"] = {}
     return manifest

core/dbt/contracts/graph/metrics.py

@@ -1,15 +1,8 @@
-from dbt.contracts.graph.manifest import Manifest, Metric
+from dbt.node_types import NodeType
-from dbt_semantic_interfaces.type_enums import MetricType
-
-from typing import Any, Dict, Iterator, List
-
-
-DERIVED_METRICS = [MetricType.DERIVED, MetricType.RATIO]
-BASE_METRICS = [MetricType.SIMPLE, MetricType.CUMULATIVE, MetricType.CONVERSION]
 
 
 class MetricReference(object):
-    def __init__(self, metric_name, package_name=None) -> None:
+    def __init__(self, metric_name, package_name=None):
         self.metric_name = metric_name
         self.package_name = package_name
 
@@ -24,74 +17,76 @@ class ResolvedMetricReference(MetricReference):
     for working with metrics (ie. __str__ and templating functions)
     """
 
-    def __init__(self, node: Metric, manifest: Manifest) -> None:
+    def __init__(self, node, manifest, Relation):
         super().__init__(node.name, node.package_name)
         self.node = node
         self.manifest = manifest
+        self.Relation = Relation
 
-    def __getattr__(self, key) -> Any:
+    def __getattr__(self, key):
         return getattr(self.node, key)
 
-    def __str__(self) -> str:
+    def __str__(self):
         return f"{self.node.name}"
 
     @classmethod
-    def parent_metrics(cls, metric_node: Metric, manifest: Manifest) -> Iterator[Metric]:
+    def parent_metrics(cls, metric_node, manifest):
-        """For a given metric, yeilds all upstream metrics."""
         yield metric_node
 
         for parent_unique_id in metric_node.depends_on.nodes:
-            node = manifest.expect(parent_unique_id)
+            node = manifest.metrics.get(parent_unique_id)
-            if isinstance(node, Metric):
+            if node and node.resource_type == NodeType.Metric:
                 yield from cls.parent_metrics(node, manifest)
 
     @classmethod
-    def parent_metrics_names(cls, metric_node: Metric, manifest: Manifest) -> Iterator[str]:
+    def parent_metrics_names(cls, metric_node, manifest):
-        """For a given metric, yeilds all upstream metric names"""
+        yield metric_node.name
-        for metric in cls.parent_metrics(metric_node, manifest):
+
-            yield metric.name
+        for parent_unique_id in metric_node.depends_on.nodes:
+            node = manifest.metrics.get(parent_unique_id)
+            if node and node.resource_type == NodeType.Metric:
+                yield from cls.parent_metrics_names(node, manifest)
 
     @classmethod
-    def reverse_dag_parsing(
+    def reverse_dag_parsing(cls, metric_node, manifest, metric_depth_count):
-        cls, metric_node: Metric, manifest: Manifest, metric_depth_count: int
+        if metric_node.calculation_method == "derived":
-    ) -> Iterator[Dict[str, int]]:
-        """For the given metric, yeilds dictionaries having {<metric_name>: <depth_from_initial_metric} of upstream derived metrics.
-
-        This function is intended as a helper function for other metric helper functions.
-        """
-        if metric_node.type in DERIVED_METRICS:
             yield {metric_node.name: metric_depth_count}
+            metric_depth_count = metric_depth_count + 1
 
-            for parent_unique_id in metric_node.depends_on.nodes:
+        for parent_unique_id in metric_node.depends_on.nodes:
-                node = manifest.expect(parent_unique_id)
+            node = manifest.metrics.get(parent_unique_id)
-                if isinstance(node, Metric):
+            if (
-                    yield from cls.reverse_dag_parsing(node, manifest, metric_depth_count + 1)
+                node
+                and node.resource_type == NodeType.Metric
+                and node.calculation_method == "derived"
+            ):
+                yield from cls.reverse_dag_parsing(node, manifest, metric_depth_count)
 
     def full_metric_dependency(self):
-        """Returns a unique list of all upstream metric names."""
         to_return = list(set(self.parent_metrics_names(self.node, self.manifest)))
         return to_return
 
-    def base_metric_dependency(self) -> List[str]:
+    def base_metric_dependency(self):
-        """Returns a unique list of names for all upstream non-derived metrics."""
         in_scope_metrics = list(self.parent_metrics(self.node, self.manifest))
-        base_metrics = {
-            metric.name for metric in in_scope_metrics if metric.type not in DERIVED_METRICS
-        }
 
-        return list(base_metrics)
+        to_return = []
+        for metric in in_scope_metrics:
+            if metric.calculation_method != "derived" and metric.name not in to_return:
+                to_return.append(metric.name)
 
-    def derived_metric_dependency(self) -> List[str]:
+        return to_return
-        """Returns a unique list of names for all upstream derived metrics."""
+
+    def derived_metric_dependency(self):
         in_scope_metrics = list(self.parent_metrics(self.node, self.manifest))
-        derived_metrics = {
-            metric.name for metric in in_scope_metrics if metric.type in DERIVED_METRICS
-        }
 
-        return list(derived_metrics)
+        to_return = []
+        for metric in in_scope_metrics:
+            if metric.calculation_method == "derived" and metric.name not in to_return:
+                to_return.append(metric.name)
 
-    def derived_metric_dependency_depth(self) -> List[Dict[str, int]]:
+        return to_return
-        """Returns a list of {<metric_name>: <depth_from_initial_metric>} for all upstream metrics."""
+
+    def derived_metric_dependency_depth(self):
         metric_depth_count = 1
         to_return = list(self.reverse_dag_parsing(self.node, self.manifest, metric_depth_count))