Jeremy Cohen
75782a88ae
Try dbtRelation class
2022-09-29 12:46:36 +02:00
Gerda Shank
3562637984
Remove parsers/source.py type ignores ( #5953 )
2022-09-28 11:24:59 -04:00
Callum McCann
17aca39e1c
Adding metric expression validation ( #5873 )
...
* adding validation
* changie
* code formatting
* updating for review
* updating tests
2022-09-27 12:38:03 -04:00
Yoshiaki Ishihara
59744f18bb
Fix typos of comments in core/dbt/adapters/ ( #5693 )
2022-09-27 09:10:24 -07:00
Rachel
f1326f526c
Runtime: Prevent introspective queries at compile (SL only) ( #5926 )
...
* Preliminary changes to keep compile from connecting to the warehouse for runtime calls
* Adds option to lib to skip connecting to warehouse for compile; adds prelim tests
* Removes unused imports
* Simplifies test and renames to SqlCompileRunnerNoIntrospection
* Updates name in tests
* Spacing
* Updates test to check for adapter connection call instead of compile and execute
* Removes commented line
* Fixes test names
* Updates plugin to postgres type as snowflake isn't available
* Fixes docstring
* Fixes formatting
* Moves conditional logic out of class
* Fixes formatting
* Removes commented line
* Moves import
* Unmoves import
* Updates changelog
* Adds further info to method docstring
2022-09-27 09:49:55 -04:00
Jeremy Cohen
834ac716fd
Prefer internal macros when called explicitly ( #5907 )
...
* Add functional test
* Prefer internal macros when called explicitly
* Add changelog entry
* update tests format
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com >
2022-09-27 08:50:55 -04:00
Doug Beatty
0487b96098
Array macros ( #5823 )
...
* Helper macro to cast from array to string
* Default implementations and tests for array macros
* Trim Trailing Whitespace
* Changelog entry
* Remove dependence upon `cast_array_to_string` macro
* pre-commit fixes
* Remove `cast_array_to_string` macro
* pre-commit fix
* Trivial direct test; array_concat/append test non-triviallly indirectly
* Remove vestigial `lstrip`
2022-09-26 13:40:15 -06:00
FishtownBuildBot
dbd36f06e4
Add new index.html and changelog yaml files from dbt-docs ( #5925 )
2022-09-26 14:00:04 -05:00
dave-connors-3
38ada8a68e
merge exclude columns for incremental models ( #5457 )
...
* exlcude cols like in dbt_utils.star
* dispatch macro
* changelog entry
Co-authored-by: Gerda Shank <gerda@dbtlabs.com >
2022-09-26 14:49:12 -04:00
Doug Beatty
e58edaab2d
Test for Koalas DataFrames ( #5928 )
2022-09-26 12:41:56 -06:00
Doug Beatty
c202e005cd
Tests for current_timestamp ( #5935 )
...
* Tests for `current_timestamp`
* Black formatting
2022-09-26 12:31:36 -06:00
Peter Webb
8129862b3c
CT-1221 add handle to changie ( #5923 )
...
* Add 'peterallenwebb' to changie's core_team list.
* fix accidental line break
2022-09-26 11:44:20 -04:00
Drew Banin
4e8aa007cf
Fix adapter reset race condition in lib.py ( #5921 )
...
* (#5919 ) Fix adapter reset race condition in lib.py
* run black
* changie
2022-09-26 10:26:20 -04:00
Doug Beatty
fe88bfabbf
Click CLI profiles directory ( #5896 )
...
* Default directories for projects and profiles
* Re-write of get_nearest_project_dir()
* Trim Trailing Whitespace
* Functionally equivalent resolvers
2022-09-24 10:47:47 -06:00
Gerda Shank
5328a64df2
CT 815 partial parsing handling of deleted metrics ( #5920 )
...
* Update delete_schema_metric to schedule referencing nodes for reparsing
* Changie
2022-09-23 18:48:00 -04:00
Chenyu Li
87c9974be1
improve error message for parsing args ( #5895 )
...
* improve error message for parsing args
* update error message
* Update models.py
* skip stack_trace for all dbt Exceptions
2022-09-23 13:31:10 -07:00
Emily Rockman
f3f509da92
update disabled metrics/exposures to use add_disabled ( #5909 )
...
* update disabled metrics/exposures to use add_disabled
* put back func fo tests
* add pp logic
* switch elif
* fix test name
* return node
2022-09-23 13:24:37 -05:00
dave-connors-3
5e8dcec2c5
update flag to -f ( #5908 )
2022-09-23 20:20:29 +02:00
Chenyu Li
56783446db
PySpark dataframe related tests ( #5906 )
...
Co-authored-by: Doug Beatty <44704949+dbeatty10@users.noreply.github.com >
2022-09-22 16:27:27 -07:00
Chenyu Li
207cc0383d
initial support for a .dbtignore ( #5897 )
2022-09-22 09:06:35 -07:00
leahwicz
49ecd6a6a4
Adding missing permissions on GHA ( #5870 )
...
* Adding missing permissions on GHA
* Adding read all permissions explicitly
2022-09-21 14:53:27 -04:00
dave-connors-3
c109f39d82
add optional shorthand to full refresh command ( #5879 )
...
* add optional shorthand to full refresh command
* changie
* replace full refresh with shorthand in functional test
2022-09-21 14:37:28 -04:00
Ian Knox
fd778dceb5
Profiling and Adapter management work with Click ( #5892 )
2022-09-21 11:23:26 -05:00
Emily Rockman
e402241e0e
Validate exposure names and add label ( #5844 )
...
* first pass
* add label and name validation
* changelog
* fix tests
* convert ParsingError to Deprecation
* fix bug where label did not end up in parsed node
* update deprecation msg
2022-09-21 10:24:44 -05:00
Emily Rockman
a6c37c948d
Add name validation for metrics ( #5841 )
...
* add tests, add name validation
* tweak test
* small update
* change to 250 char limit, tweak message
2022-09-21 09:31:13 -05:00
Daniel Messias
fd886cb7dd
ConfigSelectorMethod should check for bools ( #5889 )
...
* ConfigSelectorMethod should check for bools
* Add changelog entry
* Add support for lists and test cases
* Typo and formatting in test
* pre-commit linting
2022-09-21 10:20:14 -04:00
James McNeill
b089a471b7
implement type_boolean macro ( #5875 )
...
* implement type_boolean macro
* changie result
2022-09-20 19:24:25 -06:00
Emily Rockman
ae294b643b
Manifest deserialization error with disabled models ( #5891 )
...
* account for disabled nodes in renaming attributes
* tweak naming
2022-09-20 14:27:25 -05:00
colin-rogers-dbt
0bd6df0d1b
[CT-1101] 024_custom_schema_tests ( #5828 )
...
* create functional custom schema tests
* delete old integration tests
* changie log
* formatting fixes
* delete changie entry and errant comment
2022-09-20 10:42:07 -07:00
Gerda Shank
7b1d61c956
Ct 1191 event history cleanup ( #5858 )
...
* Change Exceptions in events to strings. Refactor event_buffer_handling.
* Changie
* fix fire_event call MainEncounteredError
* Set EventBufferFull message when event buffer >= 10,000
2022-09-20 12:44:33 -04:00
Chenyu Li
646a0c704f
restrict python submission ( #5822 )
...
Co-authored-by: Gerda Shank <gerda@dbtlabs.com >
2022-09-20 09:39:20 -07:00
Doug Beatty
bbf4fc30a5
Default to current working directory for profiles.yml and fall back to ~/.dbt ( #5717 )
...
* Method for capturing standard out during testing (rather than logs)
* Allow dbt exit code assertion to be optional
* Verify priority order to search for profiles.yml configuration
* Updates after pre-commit checks
* Test searching for profiles.yml within the dbt project directory before `~/.dbt/`
* Refactor `dbt debug` to move to the project directory prior to looking up profiles directory
* Search the current working directory for profiles.yml
* Changelog
* Formatting with Black
* Move `run_dbt_and_capture_stdout` into the test case
* Update CLI help text
* Unify separate DEFAULT_PROFILES_DIR definitions
* Remove unused PROFILE_DIR_MESSAGE
* Remove unused DEFAULT_PROFILES_DIR
* Use shared definition of DEFAULT_PROFILES_DIR
* Define global vs. local profiles location and dynamically determine the default
* Restore original
* Remove function for determining the default profiles directory
2022-09-20 08:44:15 -04:00
jared-rimmer
6baaa2bcb0
Add metadata env method to provider context ( #5794 )
...
* Add dbt_metadata_envs contextproperty to ProviderContext
* Refactor helper methods into functions
* Fix code quality failures
* Add Changelog
2022-09-20 08:41:44 -04:00
Chenyu Li
13a595722a
add support for custom file ending ( #5845 )
2022-09-19 13:58:38 -07:00
Sam Debruyn
3680b6ad0e
remove source quoting setting in adapter tests ( #5839 )
...
* remove source quoting setting in adapter tests
* changelog entry
* formatting
2022-09-19 11:00:12 -07:00
Ian Knox
4c29d48d1c
Flags work with Click ( #5790 )
2022-09-19 12:36:38 -05:00
Chenyu Li
e00eb9aa3a
fix multiple dbt.config.get in python model ( #5850 )
...
* fix multiple dbt.config.get in python model
Co-authored-by: Stu Kilgore <stuart.kilgore@gmail.com >
2022-09-16 15:07:59 -07:00
Jeremy Cohen
f5a94fc774
Back/fwd compatibility for renamed metrics attributes ( #5825 )
...
* Naive handling for metric attr renames
* Add tests for bwd/fwd compatibility
* Add deprecation
* Add changelog entry
* PR feedback
* Small fixups
* emmyoop's suggestions
Co-authored-by: Callum McCann <cmccann51@gmail.com >
2022-09-16 11:21:35 +02:00
Sam Debruyn
b98af4ce17
remove key as reserved keyword from test_bool_or ( #5818 )
2022-09-15 09:44:25 -05:00
Matthew McKnight
b0f8d3d2f1
[CT-1100] 021_test_concurrency test conversion ( #5753 )
...
* init push for 021_test_concurrency conversion
* ref to self, delete old integration tests, core passing locally
* creating base class to send setup to snowflake
* making changes to store all setup in core, todo: remove util changes after 1050 is merged
* swap sql seeds to csv
* white space removal
* rewriting seed to see if it fixes issue in snowflake
* attempt to rewrite file for test in snowflake
* update to main
* remove unneeded variable to seeds
* remove unneeded snowflake specific code
2022-09-14 15:49:36 -05:00
Emily Rockman
6c4577f44e
Add config to disable metrics/exposures ( #5815 )
...
* first pass adding disabled functionality to metrics and exposures
* first pass at getting metrics disabled
* add unsaved file
* fix up comments
* Delete tmp.csv
* fix test
* add exposure logic, fix merge from main
* change when nodes are added to manifest, finish tests
* add changelog
* removed unused code
* minor cleanup
2022-09-14 14:35:38 -05:00
Matthew McKnight
89ee5962f5
[CT-1050] convert 020_ephemeral_test ( #5699 )
...
* init file creation for test_ephemeral conversion
* creating base class to run seed through and pass along to classes to test against
* laid out basic flow of tests, need to finish by figuring out how to handle the assertTrue sections and fix error thats occuring
* added creation and comparison of sql and expected result, seeing issue with extra appended test_ on some and issue with errorhandling regarding expect pass
* working on fixing view structure
* update to expected_sql file
* update to expected_sql file
* directory rename, close on all tests need to fix the test_test_ name change for first two tests and figure out why the new test is calling error instead of skipped in status
* renamed expected_sql to include the test_test_ephemeral style name, organized how models are imported into test classes
* move ephemeral functional test to adapter zone
* trying to include the BaseEphemeralMulti class to send to snowflake
* trying to fix snowflake test
* trying to fix snowflake test
* creation of second Base class to feed into others for testing purposes
* found way to check type of warehouse to make data type change for snowflake
* move seed into fixture, to be able to import it from core for adapter tests
* convert to csv and get test passing in core
* remove snowflake specific stuff from util
* remove whitespace
* update to main
2022-09-14 12:00:02 -05:00
Ian Knox
a096202b28
Complete CLI modeling for Click ( #5789 )
2022-09-14 10:27:47 -05:00
Stu Kilgore
7da7c2d692
Convert default selectors tests to pytest ( #5820 )
2022-09-13 11:47:13 -05:00
dependabot[bot]
1db48b3cca
Bump python from 3.10.6-slim-bullseye to 3.10.7-slim-bullseye in /docker ( #5805 )
...
* Bump python from 3.10.6-slim-bullseye to 3.10.7-slim-bullseye in /docker
Bumps python from 3.10.6-slim-bullseye to 3.10.7-slim-bullseye.
---
updated-dependencies:
- dependency-name: python
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
* Add automated changelog yaml from template for bot PR
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
2022-09-12 10:58:24 -07:00
Chenyu Li
567847a5b0
add a base python job helper class ( #5802 )
...
* add a base python job helper class
* fix comment, add changelog
* fix quote for adapters
2022-09-12 10:44:19 -07:00
Sam Debruyn
9894c04d38
fix dead anchor link in PR template ( #5814 )
...
* fix dead anchor link in PR template
The link did not go to the anchor directly, now it does
* changelog entry
2022-09-12 11:08:30 -04:00
Callum McCann
b26280d1cf
Altering Window Metric Attribute To Match Freshness Tests ( #5793 )
...
* changing window spec
* more updates
* adding to v7 json?
* chenyu rules
* updating for formatting
* updating metric deferral test
2022-09-09 16:33:32 -07:00
Callum McCann
cfece2cf51
Renaming Attributes In Metric Spec ( #5775 )
...
* making updates - see what fails
* updating tests
* adding timestamp to ok_metric_no_model
* adding changie and fixing description error
* test fixes
* updating schema renderer
* fixing test_yaml_render
* file cleaning and window tests
2022-09-09 14:59:52 -04:00
Stu Kilgore
79da002c3c
Fix warnings as errors during tests ( #5800 )
...
Added RunResultWarningMessage event to support this change.
2022-09-09 12:50:15 -05:00
Bertjan Broeksema
e3f827513f
Remove tmp file after test passed ( #5749 )
...
* Remove tmp file after test passed
* Add changelog entry
2022-09-09 11:14:43 -04:00
Gerda Shank
10b2a7e7ff
Convert test/integration/074_postgres_unlogged_table_tests ( #5752 )
...
* Convert test/integration//074_postgres_unlogged_table_tests
* Remove old test
2022-09-08 16:54:03 -04:00
jared-rimmer
82c8d6a7a8
Add invocation_args_dict to ProviderContext ( #5782 )
...
* Add invocation_args_to_dict to ProviderContext
* Change invocation_args_to_dict contextproperty to invocation_args_dict
* Fix invocation_args_dict builtin test
* Add CHANGELOG entry
* Fix formatting
2022-09-08 15:31:40 -04:00
Gerda Shank
c994717cbc
Call build_flat_graph in merge_from_artifact ( #5786 )
2022-09-08 15:30:26 -04:00
Callum McCann
e3452b9a8f
Add Window Attribute for Metrics ( #5722 )
...
* file changes
* changing to window
* adding test
* adding changie for feature
* fixing commits
* fixing tests
* adding timestamp
* fixing graph unparsed
* changing default value
2022-09-07 10:45:08 -04:00
Aram Panasenco
e95e36d63b
Include py.typed in MANIFEST.in ( #5703 )
...
This enables packages that install dbt-core from pypi to use mypy.
2022-09-06 22:03:12 -04:00
jared-rimmer
74f7416144
Add extra rm command in make clean to remove all .coverage files ( #5759 )
...
* Add extra rm command in make clean to remove all .coverage files
* Add Changie entry
2022-09-06 21:36:35 -04:00
Mila Page
1feeb804f4
Update makefile to match our CI ( #5763 )
...
* Add structured logging test and provide CI env vars to integration conditionally.
* Add the crazy inline if make feature and ax unneeded variable
Co-authored-by: Mila Page <versusfacit@users.noreply.github.com >
2022-09-06 15:42:50 -07:00
Mila Page
0f6e4f0e32
Ct 866/migrate hook tests fixed ( #5760 )
...
* Finish converting first test file.
* Finish test conversion.
* Remove old integration hook tests.
* Move location of schema.yml to models directory.
* fix snapshot delete test that was failing
* Add the extra env var check for our CI.
* Add changelog
* Remove naive json flag check and instead force all integration tests to check for environment variables using flag routine.
* Revise the changelog to be more of an explanation.
Co-authored-by: Mila Page <versusfacit@users.noreply.github.com >
2022-09-06 12:22:29 -07:00
Emily Rockman
2b44c2b456
Convert experimental parser tests ( #5772 )
...
* WIP
* fixed up tests
* remove old test
2022-09-06 14:11:22 -05:00
leahwicz
2bb31ade39
Update release-branch-tests.yml ( #5767 )
2022-09-06 14:46:53 -04:00
Mila Page
0ce12405c0
Move docs in from Jinja. ( #5762 )
...
Co-authored-by: Mila Page <versusfacit@users.noreply.github.com >
2022-09-06 10:05:05 -07:00
dependabot[bot]
b8c13e05db
Bump black from 22.6.0 to 22.8.0 ( #5750 )
...
* Bump black from 22.6.0 to 22.8.0
Bumps [black](https://github.com/psf/black ) from 22.6.0 to 22.8.0.
- [Release notes](https://github.com/psf/black/releases )
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md )
- [Commits](https://github.com/psf/black/compare/22.6.0...22.8.0 )
---
updated-dependencies:
- dependency-name: black
dependency-type: direct:development
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
* Add automated changelog yaml from template for bot PR
* Remove newline
* Delete Dependency-20220901-154946.yaml
* Add automated changelog yaml from template for bot PR
* Delete Dependency-20220906-132643.yaml
* Add automated changelog yaml from template for bot PR
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com >
2022-09-06 11:31:13 -05:00
Emily Rockman
64268d2f9b
Fix label selection syntax and trigger ( #5754 )
...
* Fix label selection syntax and trigger
* update version
2022-09-06 08:21:21 -05:00
Jeremy Cohen
8c8be68701
Roadmap update (Aug 2022) ( #5748 )
...
* Add dbt Core roadmap as of August 2022
* Cody intro
* Florian intro
* Lint my markdown
* add blurb on 1.5+ for Python next steps
* Revert "add blurb on 1.5+ for Python next steps"
This reverts commit 1659a5a727 .
* PR feedback, self review
Co-authored-by: Cody Peterson <cody.dkdc2@gmail.com >
Co-authored-by: Florian Eiden <florian.eiden@dbtlabs.com >
2022-09-01 00:31:51 +02:00
Doug Beatty
1df713fee9
Test priority order of profiles directory configuration ( #5715 )
...
* Method for capturing standard out during testing (rather than logs)
* Allow dbt exit code assertion to be optional
* Verify priority order to search for profiles.yml configuration
* Updates after pre-commit checks
* Move `run_dbt_and_capture_stdout` into the test case
2022-08-30 17:18:17 -06:00
Gerda Shank
758afd4071
ADR for why we're using betterproto for protobuf ( #5726 )
2022-08-30 12:58:44 -04:00
Jeremy Cohen
0f9200d356
Update team ownership ( #5694 )
2022-08-30 15:24:34 +02:00
github-actions[bot]
5f59ff1254
Bumping version to 1.3.0b2 and generate changelog ( #5724 )
...
* Bumping version to 1.3.0b2 and generate CHANGELOG
* Remove newlines
* Remove newline
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
2022-08-29 11:11:35 -04:00
dependabot[bot]
49e7bdbef9
Bump mashumaro[msgpack] from 3.0.3 to 3.0.4 in /core ( #5649 )
...
* Bump mashumaro[msgpack] from 3.0.3 to 3.0.4 in /core
Bumps [mashumaro[msgpack]](https://github.com/Fatal1ty/mashumaro ) from 3.0.3 to 3.0.4.
- [Release notes](https://github.com/Fatal1ty/mashumaro/releases )
- [Commits](https://github.com/Fatal1ty/mashumaro/compare/v3.0.3...v3.0.4 )
---
updated-dependencies:
- dependency-name: mashumaro[msgpack]
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
* Add automated changelog yaml from template for bot PR
* Remove newline
* Remove newline
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
2022-08-25 09:00:46 -04:00
Stu Kilgore
5466fa5575
Add supported languages to materializations ( #5695 )
...
* Add supported languages to materializations
* Add changie entry
* Linting
* add more error and only get supported language for materialization macro, update schema
* fix test and add more check
Co-authored-by: Chenyu Li <chenyu.li@dbtlabs.com >
2022-08-24 12:08:16 -07:00
dependabot[bot]
f8f21ee707
Bump python from 3.10.5-slim-bullseye to 3.10.6-slim-bullseye in /docker ( #5623 )
...
* Bump python from 3.10.5-slim-bullseye to 3.10.6-slim-bullseye in /docker
Bumps python from 3.10.5-slim-bullseye to 3.10.6-slim-bullseye.
---
updated-dependencies:
- dependency-name: python
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
* Add automated changelog yaml from template for bot PR
* Update Dependency-20220808-132327.yaml
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
Co-authored-by: Chenyu Li <chenyu.li@dbtlabs.com >
2022-08-24 11:20:00 -07:00
Chenyu Li
436737dde5
more complex visit_Call to parse chained command ( #5677 )
...
* more complex visit_Call
* add changelog
* traversing all of the tree
2022-08-24 08:02:19 -07:00
Jeremy Cohen
7f8d9a7af9
Check dbt-core version requirements when installing Hub packages ( #5651 )
...
* First cut at checking version compat for hub pkgs
* Account for field rename
* Add changelog entry
* Update error message
* Fix unit test
* PR feedback
* Try fixing test
* Edit exception msg
* Expand unit test to include pkg prerelease
* Update core/dbt/deps/registry.py
Co-authored-by: Doug Beatty <44704949+dbeatty10@users.noreply.github.com >
Co-authored-by: Doug Beatty <44704949+dbeatty10@users.noreply.github.com >
2022-08-19 23:13:03 +02:00
Doug Beatty
d80de82316
Update "Homepage" link for dbt-tests-adapter on PyPI ( #5678 )
2022-08-18 06:27:30 -06:00
Mila Page
0d02446e07
Change postgres name truncation logic to be overridable. ( #5656 )
...
* Change postgres name truncation logic to be overridable. Add exception with debugging instructions.
* Add changelog.
Co-authored-by: Mila Page <versusfacit@users.noreply.github.com >
2022-08-18 00:52:43 -07:00
Chenyu Li
a9e71b3907
fix multiple args for ref and source ( #5635 )
...
* fix multiple args for ref and source
* add test for support multi part ref and source
2022-08-17 14:13:32 -07:00
Nathaniel May
739fb98d0e
Print more information on log line interop test failures ( #5659 )
...
Print more information on log line interop test failures
2022-08-16 21:30:10 +01:00
Elize Papineau
348769fa80
Fix/incremental column precision changes ( #5395 )
...
* Only consider schema change when column cannot be expanded
* Add test for column shortening
* Add changelog entry
* Move test from integration to adapter tests
* Remove print statement
* add on_schema_change
2022-08-12 14:54:29 -07:00
ilanbenb
7efb6ab62d
Incremental model show reason for on_schema_change fail failures ( #5505 )
...
* show reason for schema change failures
When the incremental model fails, I do not get the context I need to easily fix my discrepency.
Adding more info
* Update on_schema_change.sql
Fix identation
* Added changie changes
Added changie changes
* Update on_schema_change.sql
Trim whitespaces
* Update on_schema_change.sql
Log message text enhancement
2022-08-12 10:07:30 -07:00
Vyacheslav
a3b018fd3b
Extended validations for the project names ( #5620 )
2022-08-10 16:45:42 -07:00
varun-dc
4d6208be64
Use sys.exit instead of exit ( #5627 )
...
* Use sys.exit instead of exit
* Add changelog
2022-08-09 10:22:36 -07:00
Gerda Shank
3aab9befcf
CT-729 Include schema model config in unrendered config ( #5344 )
...
* Pass patch_config_dict to build_config_dict when creating
unrendered_config
* Add test case for unrendered_config
* Changie
* formatting, fix test
* Fix test so unrendered config includes docs config
2022-08-09 12:49:29 -04:00
Léon Stefani
e5ac9df069
Fix postgres handling for unlimited varchars ( #5292 )
...
* Fix postgres handling for unlimited varchars
* fix: correctly name varchar
* chore: added changelog entry
* Update .changes/unreleased/Fixes-20220523-103843.yaml
Co-authored-by: Emily Rockman <ebuschang@gmail.com >
Co-authored-by: Emily Rockman <ebuschang@gmail.com >
2022-08-09 09:59:58 -05:00
Emily Rockman
34960d8d61
link changelog to dbt-docs for Docs kind ( #5628 )
...
* first pass
* tweaks
* convert to use dbt-docs links in contributors section
* fix eq check
* fix format of contributos prs
* update docs changelog to point back to dbt-docs
* update beta 1.3 docs changelog
* remove optional param
* make issue inclusion conditional on being filled
2022-08-09 09:19:40 -05:00
Callum McCann
94a7cfa58d
Adding Metric Helper Functions ( #5607 )
...
* Adding helper functions
* adding bad test
* use ResolvedMetricReference
* adding pytest
* adding changie updates
* adding pre-commit changes
Co-authored-by: Chenyu Li <chenyu.li@dbtlabs.com >
2022-08-08 08:37:09 -05:00
Gerda Shank
eb72dbf32a
Do not render metrics description field when doing render_data ( #5603 )
2022-08-05 09:19:46 -04:00
Emily Rockman
9eb411f7b7
bring in new index and add changelog ( #5614 )
...
* bring in new index and add changelog
* Update Docs-20220804-134138.yaml
2022-08-04 14:07:18 -05:00
Benoit Perigaud
32415e3659
Add docs as a real node config and support node_color for coloring the DAG ( #5397 )
...
* add Optional node_color config in Docs dataclass
* Remove node_color from the original docs config
* Add docs config and input validation
* Handle when docs is both under docs and config.docs
* Add node_color to Docs
* Make docs a Dict to avoid parsing errors
* Make docs a dataclass instead of a Dict
* Fix error when using docs as dataclass
* Simplify generator for the default value
* skeleton for test fixtures
* bump manifest to v7
* + config hierarchy tests
* add show override tests
* update manifest
* Remove node_color from the original docs config
* Add node_color to Docs
* Make docs a Dict to avoid parsing errors
* Make docs a dataclass instead of a Dict
* Simplify generator for the default value
* + config hierarchy tests
* add show override tests
* Fix unit tests
* Add tests in case of incorrect input for node_color
* Rename tests and Fix typos
* Fix functional tests
* Fix issues with remote branch
* Add changie entry
* modify tests to meet standards (#5608 )
Co-authored-by: Matt Winkler <matt.winkler@fishtownanalytics.com >
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com >
2022-08-03 14:31:30 -05:00
Gerda Shank
7886924c07
Comment out line to generate new manifest in test_previous_version_state.py ( #5604 )
2022-08-03 14:18:48 -04:00
Emily Rockman
40b55ed65a
convert to reusable action ( #5565 )
...
* convert to reusable action
* fix branch name
* reimplemented changelog
* update to use workflow
* fix typo
* move def
* inherit secrets
* send in comment/label
* specify GITHUB_TOKEN
* Add automated changelog yaml from template for bot PR
* Delete Dependency-20220801-193810.yaml
* Add automated changelog yaml from template for bot PR
* remove dummy changelog
* remove token
* rename file
* point to main
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
2022-08-03 11:33:11 -05:00
Emily Rockman
4f5b9e686c
Store when default env vars are used in manifest ( #5589 )
...
* WIP
* handle defauly env vars
* fix typo
* add changelog
* small fixes
* add constants.py file
2022-08-02 20:53:11 -05:00
Emily Rockman
95284aff68
rename .github readme ( #5597 )
2022-08-01 19:22:26 -05:00
kadero
063ff9c254
Add defer to docs and compile commands ( #4514 )
...
* Add defer to docs and compile commands
Add docs generate defer test
Fix CLA check
Add compile defer test
* Update changelog
* Restore changelog
* Add changie entry
* Move defer_to_manifest to CompileTask
* Add check to to verify defer works as expected
* Add assert
2022-08-01 14:38:08 -07:00
Vyacheslav
26b33e668d
use MethodName.File when value ends with .csv ( #5581 )
...
* [CT-953] [Feature] Support for .csv files when using file selectors(for seed)
* Added Changelog entry for issue-5578
* Added Unit tests
2022-08-01 14:32:58 -07:00
Jeremy Cohen
26ac9d57d0
Convert git deps to local, faster tests ( #5515 )
2022-07-30 14:10:57 +02:00
github-actions[bot]
7bd861a351
Bumping version to 1.3.0b1 and generate changelog ( #5582 )
...
* Bumping version to 1.3.0b1 and generate CHANGELOG
* Update Docker versions for adapters
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
2022-07-29 11:59:12 -04:00
Chenyu Li
15c97f009a
fix passing in Rlock error in rpc ( #5580 )
2022-07-29 07:46:08 -07:00
Leo Folsom
5153023100
use MethodName.File when value ends with .py ( #5295 )
2022-07-29 07:33:53 -07:00
Emily Rockman
c879083bc9
add info on basics of using actions ( #5576 )
2022-07-29 08:20:00 -05:00
Chenyu Li
05bf27c958
add changelog from dbt-docs ( #5577 )
2022-07-28 15:49:35 -07:00
Chenyu Li
a7ff003d4f
python model beta feature( #5421 )
...
* Python model beta version with update to manifest that renames `raw_sql` and `compiled_sql` to `raw_code` and `compiled_code`
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
Co-authored-by: Ian Knox <ian.knox@dbtlabs.com >
Co-authored-by: Stu Kilgore <stuart.kilgore@gmail.com >
2022-07-28 11:42:59 -07:00
leahwicz
2547e4f55e
Add changelog and whitespace fix to version bump Action ( #5563 )
...
* Add changelog and whitespace fix to version bump Action
* Fixing whitespace
* Remove tabs
* Update .github/workflows/version-bump.yml
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com >
* Update .github/workflows/version-bump.yml
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com >
* Update .github/workflows/version-bump.yml
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com >
* Update .github/workflows/version-bump.yml
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com >
* Updating per comments
* Fix whitespace
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com >
2022-07-28 14:08:03 -04:00
Gerda Shank
b43fc76701
Fix handling of top-level exceptions ( #5560 )
2022-07-27 23:52:17 -04:00
Jeremy Cohen
48464a22a4
Proposed updates to issue "wayfinder" ( #5426 )
...
* Propose updates to issue wayfinder
* Updating bug template with clearer description
* Updated link to new troubleshooting spot
* Updating Cloud messaging
* Adding required fields and dbt versions
* Fixed whitespace
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
2022-07-27 22:25:34 -04:00
Vyacheslav
c3891d78e4
[CT-700] [Bug] Logging tons of asterisks for sensitive env vars ( #5518 )
...
* [CT-700] [Bug] Logging tons of asterisks when sensitive env vars are missing
* [CT-700][Bug] Added changelog entry
* Updated the changelog body message
2022-07-27 11:40:32 -05:00
Nicholas A. Yager
69ce6779e1
Improve CompilationException messaging for generic test Jinja rendering ( #5393 )
...
* feat: Improve generic test UndefinedMacroException message
The error message rendered from the `UndefinedMacroException` when
raised by a TestBuilder is very vague as to where the problem is
and how to resolve it. This commit adds a basic amount of
information about the specific model and column that is
referencing an undefined macro.
Note: All custom macros referenced in a generic test config will
raise an UndefinedMacroException as of v0.20.0.
* feat: Bubble CompilationException into schemas.py
I realized that this exception information would be better if
CompilationExceptions inclulded the file that raised the exception.
To that end, I created a new exception handler in `_parse_generic_test`
to report on CompilationExceptions raised during the parsing of
generic tests. Along the way I reformatted the message returned
from TestBuilder to play nicely with the the existing formatting of
`_parse_generic_test`'s exception handling code.
* feat: Add tests to confirm CompileException
I've added a basic test to confirm that the approriate
CompilationException when a custom macro is referenced
in a generic test config.
* feat: Add changie entry and tweak error msg
* Update .changes/unreleased/Under the Hood-20220617-150744.yaml
Thanks to @emmyoop for the recommendation that this be listed as a Fix change instead of an "Under the Hood" change!
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com >
* fix: Simplified Compliation Error message
I've simplified the error message raised during a Compilation Error
sourced from a test config. Mainly by way of removing tabs and newlines
where not required.
* fix: Convert format to fstring in schemas
This commit moves a format call to a multiline fstring in the
schemas.py file for CompilationExceptions.
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com >
2022-07-27 09:41:22 -05:00
Emily Rockman
a206cfce65
add readme to .github ( #5487 )
...
* add readme to .github
* more changes to readme
* improve docs
* more readme tweaks
* add more docs
* incorporate feedback
* removed section with no info
2022-07-25 16:16:36 -05:00
Emily Rockman
3f54f30349
Remove markupsafe pin ( #5507 )
...
* update markupsafe pin
* add changelog
* completely remove markupsafe pin
2022-07-25 09:47:25 -05:00
Matthew McKnight
1071a4681d
reformat changie.yaml to ignore dependabot ( #5508 )
...
* reformat changie.yaml to ignore dependabot
* Removed modifled file from PR
* Removed modifled file from PR
2022-07-22 09:45:37 -05:00
Gerda Shank
2548ba9936
Refactoring of incremental materialization ( #5359 )
2022-07-21 14:11:23 -04:00
Nathaniel May
999ed0b74c
postgres: add exponential backoff to connection retries ( #5503 )
...
add exponential backoff to postgres connection retries
2022-07-21 13:10:52 -04:00
Emily Rockman
eef7bca005
snyk automation plus bot changelog consolidation ( #5479 )
...
* first pass at snyk changelog entry
* refactor for single workflow for all bot PRs
* exclude snyk from contributors list
* point action to branch temporarily
* replace quotes
* point to released tag
2022-07-20 17:48:35 -05:00
Gerda Shank
5686cab5a0
Bump mashumaro from 2.9 to 3.0.3 in /core ( #5118 )
...
* add pyyaml as a setup requirement
2022-07-20 17:15:51 -04:00
dependabot[bot]
99bc292588
Bump mypy from 0.961 to 0.971 ( #5495 )
...
* Bump mypy from 0.961 to 0.971
Bumps [mypy](https://github.com/python/mypy ) from 0.961 to 0.971.
- [Release notes](https://github.com/python/mypy/releases )
- [Commits](https://github.com/python/mypy/compare/v0.961...v0.971 )
---
updated-dependencies:
- dependency-name: mypy
dependency-type: direct:development
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
* Add automated changelog yaml from template
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
2022-07-19 21:37:49 -04:00
Emily Rockman
a1ee348a6f
jinja2 v3 upgrade ( #5465 )
...
* update version
* ignore mypy for now
* add changelog
2022-07-19 16:16:31 -05:00
Tomás Farías Santana
2048a1af6f
[CT-472] feat: Retrying method for acquiring connection handles ( #5432 )
...
Add reusable function for retrying adapter connections. Utilize said function to add retries for Postgres (and Redshift).
2022-07-19 14:55:55 -04:00
Emily Rockman
71223dc253
add timeouts to integration tests ( #5481 )
...
* add timeout to integration tests
* add timeout to all jobs in workflow
2022-07-19 10:31:46 -05:00
Emily Rockman
e03d35a9fc
Fix: Rename try methods to strict ( #5477 ) ( #5478 )
...
* rename strict methods
* add changelog
Co-authored-by: Jeremy Yeo <jeremyyeo@users.noreply.github.com >
2022-07-15 09:38:21 -05:00
github-actions[bot]
f988f76fcc
Bumping version to 1.3.0a1 ( #5463 )
...
* Bumping version to 1.3.0a1
* Remove whitespace
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
2022-07-12 14:17:06 -04:00
Emily Rockman
0cacfd0f88
cleanup changelog after cutting 1.2.latest branch ( #5462 )
2022-07-12 11:05:50 -05:00
Matthew McKnight
c25260e5dd
Add Grant SQL Macros ( #5369 )
...
* init push or ct-660 work
* changes to default versions of get_show_grant_sql and get_grant_sql
* completing init default versions of all macros being called for look over and collaboration
* minor update to should_revoke
* post pairing push up (does have log statements to make sure we remove)
* minor spacing changes
* minor changes, and removal of logs so people can have clean grab of code
* minor changes to how get_revoke_sql works
* init attempt at applying apply_grants to all materialzations
* name change from recipents -> grantee
* minor changes
* working on making a context to handle the diff gathering between grant_config and curreent_grants to see what needs to be revoked, I know if we assign a role, and a model becomes dependent on it we can't drop the role now still not seeing the diff appear in log
* removing logs from most materializations to better track diff of grants generation logs
* starting to build out postgres get_show_grant_sql getting empty query errors hopefully will clear up as we add the other postgres versions of macros and isn't a psycopg2 issue as indicated by searching
* 6/27 eod update looking into diff_grants variable not getting passed into get_revoke_sql
* changes to loop cases
* changes after pairing meeting
* adding apply_grants to create_or_replace_view.sql
* models are building but testing out small issues around revoke statement never building
* postgrest must fixes from jeremy's feedback
* postgres minor change to standarize_grants_dict
* updating after pairing with dough and jeremey incorporating the new version of should revoke logic.
* adding ref of diff_of_two_dicts to base keys ref
* change of method type for standardize_grants_dict
* minor update trying to fix unit test
* changes based on morning feedback
* change log message in default_apply_grants macro
* CT-808 grant adapter tests (#5447 )
* Create initial test for grants
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
* rename grant[privilege] -> grant_config[privilege]
* postgres macro rename to copy_grants
* CT-808 more grant adapter tests (#5452 )
* Add tests for invalid user and invalid privilege
* Add more grant tests
* Macro touch ups
* Many more tests
* Allow easily replacing privilege names
* Keep adding tests
* Refactor macros to return lists, fix test
* Code checks
* Keep tweaking tests
* Revert cool grantees join bc Snowflake isnt happy
* Use Postgres/BQ as standard for standardize_grants_dict
* Code checks
* add missing replace
* small replace tweak, add additional dict diffs
* All tests passing on BQ
* Add type cast to test_snapshot_grants
* Refactor for DRYer apply_grants macros
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com >
* update to main, create changelog, whitespace fixes
Co-authored-by: Gerda Shank <gerda@dbtlabs.com >
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com >
2022-07-11 11:58:41 -05:00
Batuhan Taskaya
c521fa6b74
Add --target-path as a CLI option. ( #5402 )
2022-07-07 12:49:25 -04:00
Emily Rockman
f304b4b2da
add new index file for dbt-docs for 1.2.0 release ( #5446 )
...
* add new index file for dbt-docs for 1.2.0 release
* add PR number to changelogs
2022-07-06 16:12:38 -05:00
Drew Banin
064d890172
( #4884 ) Add support for ratio metrics ( #5027 )
...
* wip
* More support for ratio metrics
* Formatting and linting
* Fix unit tests
* Support disabling metrics
* mypy
* address all TODOs
* make pypy happy
* wip
* checkpoint
* refactor, remove ratio_terms
* flake8 and unit tests
* remove debugger
* quickfix for filters
* Experiment with functional testing for 'expression' metrics
* reformatting slightly
* make file and mypy fix
* remove config from metrics - wip
* add metrics back to context
* adding test changes
* fixing test metrics
* revert name audit
* pre-commit fixes
* add changelog
* Bumping manifest version to v6 (#5430 )
* Bumping manifest version to v6
* Adding manifest file for tests
* Reverting unneeded changes
* Updating v6
* Updating test to add metrics field
* Adding changelog
* add v5 to backwards compatibility
* Clean up test_previous_version_state, update for v6 (#5440 )
* Update test_previous_version_state for v6. Cleanup
* Regenerate, rm breakpoint
* Code checks
* Add assertion that will fail when we bump manifest version
* update tests to automatically tests all previous versions
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com >
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
Co-authored-by: Callum McCann <cmccann51@gmail.com >
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com >
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
2022-07-06 16:01:14 -05:00
Paolo Quadri
febbd978b5
Fix/fix quotes behaviour in snapshots ( #5389 )
...
* fix: add quotes to relation
* fix: add quotes correctly
* chore: reset main.py
* chore: reset main.py from master
* chore: newline
* chore: changie
2022-07-06 16:30:19 -04:00
dependabot[bot]
0d7e87fac6
Bump mypy from 0.942 to 0.961 ( #5337 )
...
* Bump mypy from 0.942 to 0.961
Bumps [mypy](https://github.com/python/mypy ) from 0.942 to 0.961.
- [Release notes](https://github.com/python/mypy/releases )
- [Commits](https://github.com/python/mypy/compare/v0.942...v0.961 )
---
updated-dependencies:
- dependency-name: mypy
dependency-type: direct:development
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
* Add automated changelog yaml from template
* Fixed deprecated abstractclassmethod caught by new version of mypy
* Fixed deprecated abstractclassmethod not caught by mypy
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
Co-authored-by: Ian Knox <ian.knox@dbtlabs.com >
2022-07-06 16:25:05 -04:00
Volkan Gürel
3500528506
Add inheritance to materialization macro resolution ( #5348 )
...
* Add inheritance to materialization macro resolution
* Add changelog entry
* Address PR feedback
2022-07-06 13:56:52 -04:00
Joe Markiewicz
c42221fcf3
feature/deps-regex-semver ( #5370 )
...
* feature/deps-regex-semver
* pre-commit fixes
* applying review updates
2022-07-06 10:56:07 -04:00
dependabot[bot]
f49f28c331
Update colorama requirement from <0.4.5,>=0.3.9 to >=0.3.9,<0.4.6 in /core ( #5388 )
...
* Update colorama requirement in /core
2022-07-05 14:46:56 -05:00
dependabot[bot]
dc964c43d9
Bump black from 22.3.0 to 22.6.0 ( #5420 )
...
* Bump black from 22.3.0 to 22.6.0
2022-07-05 14:02:15 -05:00
Peter Debelak
60e491b3c1
Improve pluralizations for Documentation and SqlOperation NodeTypes ( #5356 )
...
* Improve pluralizations for Documentation and SqlOperation NodeTypes
Previously these were `docss` and `sqlss` which leaves something to be
desired.
* Add changie changelog entry for pluralization change
* Slighly simplify node type pluralization tests
* Update node type names for sql and docs so they match pluralizations
2022-06-30 16:15:07 -04:00
Jeremy Cohen
3bfce2bac9
Move data type macros into dbt-core ( #5428 )
...
* Move data type macros into dbt-core
* Changelog entry
* Code quality checks. Fix type_float
2022-06-30 19:46:07 +02:00
Darin Douglass
d63ad4cd82
[CT-679] flip logic around reset color detection ( #5394 )
2022-06-29 17:28:01 -04:00
Jeremy Cohen
d5608dca32
Follow-up to #5334 ( #5382 )
...
* Follow-up to secret rendering changes
* Update changelog entries
* PR feedback
2022-06-28 10:06:12 +02:00
Mila Page
e7031f2d74
Ct 488/migrate invalid model tests ( #5391 )
...
Migrate 006 dependency tests to the new framework
2022-06-24 12:05:41 -04:00
github-actions[bot]
68a2996788
Bumping version to 1.2.0b1 ( #5406 )
...
* Bumping version to 1.2.0b1
* Add changelog
* Remove whitespace
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
Co-authored-by: Leah Antkiewicz <leah.antkiewicz@fishtownanalytics.com >
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
2022-06-24 10:10:39 -04:00
leahwicz
f5f0a7f908
Removing old script from bumpversion ( #5405 )
2022-06-24 09:42:48 -04:00
Jeremy Yeo
1cfc0851ca
[Fix]: remove duplicate key checking ( #5403 )
...
* remove duplicate key checking
* add changelog
2022-06-23 15:24:15 -04:00
Ulises Ojeda
d257d0b44c
Early return from dbt init if no available adapters ( #5366 )
...
* Exit from dbt init if no available adapters
* adding chnagie change
* fixing init_tests
2022-06-17 09:42:37 -07:00
Matthew McKnight
f8d347e5f8
deleting scaffold and create_adater_plugin.py file from scripts section of core as they a… ( #5117 )
...
* deleting scaffold and .py file from scripts section of core as they are either deprecated or will live outside of core
* adding changelog
* removing files that shouldn't be there
* update changelog to have link to new scaffold
* readding the original script file but changing its output ot be a print statement and leave comment that also points to the new scaffold
* sentence change
2022-06-16 14:00:08 -05:00
Doug Beatty
a02db03f45
Lift + shift for cross-db macros ( #5298 )
...
* Initialize lift + shift, dateadd + datediff
* Placeholder changelog for now
* Lift and shift cross-database macros, fixtures, and tests from dbt-utils
* Switch namespace from `dbt_utils` to `dbt`
* Remove unreferenced variable
* Remove conflicting definition of current_timestamp()
* Trim leading and trailing whitespace
* Apply Black formatting
* Remove unused import
* Remove references to other profiles
* Update .changes/unreleased/Features-20220518-114604.yaml
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
* Kick out the `type_*` tests
* Kick out the `type_*` macros
* Kick out the `current_timestamp` tests
* Kick out the `current_timestamp` macros
* Kick out the `current_timestamp` macros
* Kick out the `type_*` macros
* Use built-in adapter functionality for converting string datatypes
* Move comment above macro for postgres__any_value
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
2022-06-16 20:07:36 +02:00
Jeremy Cohen
6e8388c653
Declare compatibility for previous artifact versions ( #5346 )
...
* Add functional test
* Add is_compatible_version logic
* Add changelog entry
* Fix mypy
2022-06-16 12:19:18 +02:00
jared-rimmer
6572b7e0a5
Prettify duration message at end of execution ( #5364 )
...
* Prettify duration message at end of execution
2022-06-15 12:28:55 -07:00
Matthew McKnight
26bb5c3484
Adapters README (doc days of winter) ( #5332 )
...
* init docs write up
* updating based on feedback from gerda
* minor sentence structure changes
* minor changes after feedback
2022-06-15 10:18:07 -05:00
leahwicz
83f4992073
Adding scheduled CI testing Action ( #5349 )
...
* Adding scheduled CI testing Action
* Fixing malformed message
* Fixing messaging quotes
* Update to not fail fast
* Reordered branches
* Updating job name
* Removed PR trigger used for testing
2022-06-14 13:24:52 -04:00
dependabot[bot]
8392023e9f
Bump python from 3.10.3-slim-bullseye to 3.10.5-slim-bullseye in /docker ( #5367 )
...
* Bump python from 3.10.3-slim-bullseye to 3.10.5-slim-bullseye in /docker
Bumps python from 3.10.3-slim-bullseye to 3.10.5-slim-bullseye.
---
updated-dependencies:
- dependency-name: python
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
* Add automated changelog yaml from template
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
2022-06-14 11:34:42 -04:00
Matthew McKnight
309efaa141
remove script for snowflake oauth reset as its been moved to snowflake ( #5362 )
...
* remove script for snowflake oauth reset as its been moved to snowflake
* changelog entry
2022-06-14 09:45:40 -05:00
Chenyu Li
a5993fc866
lower networkx for py37 ( #5264 )
2022-06-13 12:13:32 -07:00
Chenyu Li
5b1bc72ae1
python model adr ( #5345 )
...
* add python model adr first version
2022-06-13 07:54:55 -07:00
Greg Roodt
72b6a80b07
Remove duplicate dbt script entry ( #5304 )
...
* Remove duplicate dbt script entry.
* changie
2022-06-08 16:14:37 -07:00
leahwicz
e48f7ab32e
Fixing Windows color regression ( #5327 )
...
* Fixing Windows color regression
* Cleaning up logic
* Consolidating logic to the logger
* Cleaning up vars
* Updating comment
* Removing unused import
* Fixing whitespace
* Adding changelog
2022-06-07 09:06:02 -04:00
Jeremy Cohen
16dc2be556
Update context readme, small code cleanup ( #5334 )
2022-06-06 23:39:34 +02:00
Ian Knox
eea872c319
New ancestor/descendant algo ( #5326 )
...
* New ancestor/descendant algo
* changelog
2022-06-02 10:46:01 -07:00
Gerda Shank
189c06dbb1
CT-710 fix test graph selection ( #5324 )
...
* Fix test_graph_selection so it can run separately
* Changie
2022-06-01 12:01:10 -04:00
Kyle Wigley
74662d1527
fix pip upgrade step in CI for windows ( #5320 )
...
* fix pip upgrade step in CI for windows
* Changie
Co-authored-by: Gerda Shank <gerda@dbtlabs.com >
2022-06-01 11:24:05 -04:00
Mila Page
75f3e8cb74
Add notes on invoking Python methods in adapter macros ( #5306 )
...
Co-authored-by: Mila Page <versusfacit@users.noreply.github.com >
2022-05-31 08:55:30 -07:00
Jeremy Cohen
aeee1c23a6
Tidying up materialization logic: relation names, cache loading, etc ( #5221 )
...
* Clean up existing_relation, load_relation, etc
* Add changelog entry
* Empty commit, retrigger CI
2022-05-20 18:02:14 +02:00
Gerda Shank
e50678c914
Ct-581 grants as configs ( #5230 )
...
* Handle 'grants' in NodeConfig, with correct merge behavior
* Fix a bunch of tests
* Add changie
* Actually add the test
* Change to default replace of grants with '+' extending them
* Additional tests, fix config_call_dict handling
* Tweak _add_config_call to remove unnecessary isinstance checks
2022-05-19 14:50:47 -04:00
Josh Wills
ae62f5708c
Switch the Makefile to use the default installed Python version instead of py38 for local testing ( #5269 )
...
* Setting up an env var to use to override the tox python variable used for local dev
* Switch over to py-based tox envs instead of the py38 ones to be friendly to dbt-core devs who don't work at dbt Labs
* changie
2022-05-19 12:50:26 -04:00
Gerda Shank
cda88d1948
CT-159 Remove docs file from manifest when deleting doc node ( #5270 )
...
* Convert partial parsing docs tests
* Failing test
* Remove doc file from manifest when doc node is removed
* Changie
2022-05-19 11:37:45 -04:00
Elize Papineau
e7218d3e99
Truncate relation names when appending a suffix ( #4921 )
...
* Truncate relation names when appending a suffix that will result in len > 63 characters using make_temp_relation and make_backup_relation macros
* Remove timestamp from suffix appended to backup relation
* Add changelog entry
* Implememt make_relation_with_suffix macro
* Add make_intermediate_relation macro that controls _tmp relation creation in table and view materializations to delienate from database- and schema-less behavior of relation returned from make_temp_relation
* Create backup_relation at top of materialization to use for identifier
* cleanup
* Add dstring arg to make_relation_with_suffix macro
* Only reference dstring in conditional of make_relation_with_suffix macro
* Create both a temp and intermediate relation, update preexisting_temp_relation to preexisting_intermediate_relation
* Migrate test updates to new test location
* Remove restored tmp.csv
* Revert "Remove restored tmp.csv"
This reverts commit 900c9dbcad9a1e6a5a6737c84004504bfdd9926f.
* Actually remove restored tmp.csv
2022-05-19 13:57:15 +02:00
leahwicz
2c42fb436c
Creating ADR for versioning and branching strategy ( #4998 )
...
* Creating ADR for versioning and branching strategy
* Fixing image link
* Grammar clean-up
Co-authored-by: Stu Kilgore <stuart.kilgore@gmail.com >
* Grammar clean-up
Co-authored-by: Stu Kilgore <stuart.kilgore@gmail.com >
* Update docs/arch/adr-003-versioning-branching-strategy.md
Co-authored-by: Stu Kilgore <stuart.kilgore@gmail.com >
* Update docs/arch/adr-003-versioning-branching-strategy.md
Co-authored-by: Stu Kilgore <stuart.kilgore@gmail.com >
* Update docs/arch/adr-003-versioning-branching-strategy.md
Co-authored-by: Stu Kilgore <stuart.kilgore@gmail.com >
* Update docs/arch/adr-003-versioning-branching-strategy.md
Co-authored-by: Stu Kilgore <stuart.kilgore@gmail.com >
* Update docs/arch/adr-003-versioning-branching-strategy.md
Co-authored-by: Stu Kilgore <stuart.kilgore@gmail.com >
* Update docs/arch/adr-003-versioning-branching-strategy.md
Co-authored-by: Stu Kilgore <stuart.kilgore@gmail.com >
* Update docs/arch/adr-003-versioning-branching-strategy.md
Co-authored-by: Stu Kilgore <stuart.kilgore@gmail.com >
* Update docs/arch/adr-003-versioning-branching-strategy.md
Co-authored-by: Stu Kilgore <stuart.kilgore@gmail.com >
* Updating Outside Scope section
* Changing from using type to stage
* Adding section on getting changes into certain releases
* Changed stages to phases
* Some wording updates
* New section for branching pros and cons
* Clarifying version bump statement
* A few minor comment fix ups
* Adding requirement to define released
* Updating to completed!
Co-authored-by: Stu Kilgore <stuart.kilgore@gmail.com >
2022-05-18 23:12:20 -04:00
Gerda Shank
a9e1a0e00a
Tweak test to avoid set ordering problem ( #5272 )
2022-05-18 17:28:48 -04:00
Jeremy Cohen
0d8e061a3d
Add dbt Core roadmap as of May 2022 ( #5246 )
2022-05-15 10:47:55 +02:00
Stu Kilgore
7532420eef
Fix macro modified from previous state with pkg ( #5224 )
...
* Fix macro modified from previous state with pkg
When iterating through nodes to check if any of its macro dependencies
have been modified, the state selector will first check all upstream
macro dependencies before returning a judgement.
2022-05-13 13:04:15 -05:00
Josh Wills
03b17ff401
Add support for File Selectors and add file selectors to the default method selector list ( #5241 )
...
* Add a new selector method for files and add it to the default method selection criteria if the given selector has a . in it but no path separators
* Add a file: selector method to the default selector methods because it will make Pedram happy
* changie stuff
2022-05-13 09:39:55 -07:00
Tomás Farías Santana
fc1fc2d5e9
fix: Avoid access to profile when calling str(UnsetProfileConfig) ( #5209 )
...
* fix: Avoid access to profile when calling str(UnsetProfileConfig)
dbt.config.UnsetProfileConfig inherits __str__ from
dbt.config.Project. Moreover, UnsetProfileConfig also raises an
exception when attempting to access unset profile attributes. As
Project.__str__ ultimately calls to_project_config and accesses said
profile attributes, we override to_project_config in
UnsetProfileConfig to avoid accessing the attributes that raise an
exception.
This allows calling str(UnsetProfileConfig) and
repr(UnsetProfileConfig).
Basic unit testing is also included in commit.
* fix: Skip repr for profile fields in UnsetProfileConfig
* chore(changie): Add changie file
2022-05-13 08:36:13 -07:00
Jeremy Yeo
7e43f36bb1
Feature: Add set/zip function to contexts ( #5107 )
...
* add set function to contexts
* add zip function to contexts
* add changelog
* add try_ equivalents
* remove defaults
* add tests
* update tests
2022-05-13 10:25:51 -05:00
Jeremy Cohen
72c17c4464
Fix: use adapter.get_columns_in_relation ( #5232 )
2022-05-11 16:48:25 +02:00
Jeremy Cohen
3996a69861
Fix: column comparison logic for check-strategy snapshots ( #5223 )
...
* Add test case
* Update comparison in snapshot_check_all_get_existing_columns
* Add changelog entry
2022-05-10 11:04:56 +02:00
leahwicz
aa8115aa5e
Bumping hologram version ( #5218 )
...
* Bumping hologram version
* Add automated changelog yaml from template
* Updating issue
* Loosen requirement range
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
2022-05-09 15:49:11 -04:00
Ian Knox
ab0c3510eb
Revert inaccurate mypy change ( #5216 )
2022-05-06 09:21:48 -05:00
Ian Knox
4480d05cfb
Update mypy to latest and turn on everywhere ( #5171 )
2022-05-05 12:00:11 -05:00
Gerda Shank
788694ec5b
Convert materialization tests to new test framework ( #5212 )
2022-05-04 13:55:09 -04:00
Nicolas Parot Alvarez
fb5bb7fff3
Improve tracking error logging message ( #5211 )
2022-05-04 11:00:13 -04:00
Matthew McKnight
c270a77552
propseal for modification to drop_test_schema ( #5198 )
...
* propseal for modification to drop_test_schema
* changelog
* remove hard coded run_dbt version and put back previous version of drop_test_schema, add commit to drop_schema
2022-05-03 13:10:02 -05:00
Alex Rosenfeld
a2e040f389
Allow the target name to be set in profile_template.yml ( #5184 )
...
* Allow the target name to be set in profile_template.yml
2022-05-03 11:44:40 -05:00
Michael Manganiello
a4376b96d8
seed: Add new macro get_csv_sql ( #5207 )
...
new macro `get_csv_sql`
2022-05-03 11:32:24 -05:00
Gerda Shank
ed5df342ca
Convert 013_context_vars_tests to context_methods. Move existing cli_vars into context_methods ( #5199 )
2022-05-02 16:02:37 -04:00
Emily Rockman
96f063e077
update label check ( #5194 )
2022-05-02 11:51:23 -05:00
leahwicz
ee8f81de6a
Adding Skip Changelog label to Version Bump action ( #5203 )
2022-05-02 12:40:58 -04:00
Emily Rockman
935edc70aa
remove reference to unused ok to test label ( #5149 )
2022-05-02 09:21:17 -05:00
dependabot[bot]
28c44a9be7
Bump ubuntu from 20.04 to 22.04 ( #5141 )
...
* Bump ubuntu from 20.04 to 22.04
2022-04-29 15:53:25 -05:00
Stu Kilgore
a2b3602485
Convert list tests to pytest ( #5178 )
2022-04-28 15:00:39 -05:00
Jeremy Yeo
3733817488
Fix: add warning on duplicated yaml keys ( #5146 )
...
* add warning on duplicated yaml keys
* update structure and tests
* fix old test schema file
* add changelog
2022-04-28 09:31:18 -04:00
Jeremy Cohen
c5fb6c275a
Update README for dbt-tests-adapter ( #5182 )
...
* Update README for dbt-tests-adapter
* Add logo
2022-04-28 15:26:02 +02:00
Gerda Shank
f633e9936f
When parsing 'all_sources' should be a list of unique dirs ( #5176 )
...
* When parsing 'all_sources' should be a list of unique dirs
* Changie
* Fix some unit tests of all_source_paths
* Convert 039_config_tests
* Remove old 039_config_tests
* Add test for duplicate directories in 'all_source_files'
2022-04-27 21:02:51 -04:00
Gerda Shank
4e57c51c7a
Ct-65 metrics names with spaces ( #5173 )
...
* Convert existing metrics test
* add non-failing test for names with spaces
* Raise ParsingException if metrics name contains spaces
* Remove old metrics tests
2022-04-27 10:57:32 -04:00
Gary James
6267572ba7
Fix adding new cols to check_cols in snapshots ( #4893 )
2022-04-26 18:55:14 -04:00
Daniel Diamond
32e1924c3b
Add selector method capabilities to selectors ( #4827 )
2022-04-26 18:52:08 -04:00
Chenyu Li
55af3c78d7
remove extra class and add connection test ( #5163 )
...
* remove extra class and add connection test
* add project artifact to avoid breaking other tests
* add comment
2022-04-26 16:07:23 -06:00
Mila Page
bdff19d909
migrate 009_data_tests to new test framework ( #5139 )
...
* Fold so-called 'data' test into new framework with new vocabulary to match.
* Add missing files including changelog.
* Remove unneeded Changelog per team policy on test conversions.
* Refactor test code to better use our pytest framework. Strengthen assertions.
Co-authored-by: Mila Page <versusfacit@users.noreply.github.com >
2022-04-26 14:59:27 -07:00
Ben Dowling
f87c7819fb
Add itertools to modules ( #5140 )
...
* GH hygiene: contributing guide, templates, stalebot (#4967 )
* Update contributing guide
* Update issue + PR templates
* Stalebot for all issues, no exceptions
* Update links
* Missed one
* PR feedback
* Update CHANGELOG
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
2022-04-26 16:32:06 -05:00
Mila Page
33694f3772
Ct 488/migrate invalid model tests ( #5143 )
...
* First test completed.
* Convert and update more test cases.
* Complete test migration and remove old files.
Co-authored-by: Mila Page <versusfacit@users.noreply.github.com >
2022-04-26 13:38:34 -07:00
Emily Rockman
ebfc18408b
Tweak triage-label.yml to trigger off issue labels instead of PR labels ( #5168 )
...
* fix label check
* fix label filter to go against issues not PRs
2022-04-26 15:25:20 -05:00
Emily Rockman
6958f4f12e
add triage label workflow ( #5164 )
2022-04-26 13:39:09 -05:00
Gerda Shank
1f898c859a
Use yaml renderer (with target context) for rendering selectors ( #5136 )
...
* Use yaml renderer (with target context) for rendering selectors
* Changie
* Convert cli_vars tests
* Add test for var in profiles
* Add test for cli vars in packages
* Add test for vars in selectors
2022-04-26 11:42:50 -04:00
Jeremy Cohen
ce0bcc08a6
Even more scrubbing ( #5152 )
...
* Even more scrubbing
* Changelog entry
* Even more
* remove reduendent scrub
* remove reduendent scrub
* fix encoding issue
* keep scrubbed log in args
Co-authored-by: Chenyu Li <chenyu.li@dbtlabs.com >
2022-04-26 09:35:01 -06:00
leahwicz
d1ae9dd37f
Updating backport Action permissions ( #5121 )
2022-04-25 09:33:49 -04:00
Emily Rockman
31a3f2bdee
fix retry logic failures ( #5137 )
...
* fix retry logic failures
* changelog
* add tests to make sure data is getting where it needs to
* rename file
* remove duplicate file
2022-04-25 06:08:57 -05:00
Jeremy Cohen
1390715590
GH hygiene: contributing guide, templates, stalebot ( #4967 )
...
* Update contributing guide
* Update issue + PR templates
* Stalebot for all issues, no exceptions
* Update links
* Missed one
* PR feedback
2022-04-22 13:57:15 +02:00
Doug Beatty
d09459c980
Restore ability to utilize updated_at for check_cols snapshots ( #5077 )
...
* Restore ability to configure and utilize `updated_at` for snapshots using the check_cols strategy
* Changelog entry
* Optional comparison of column names starting with `dbt_`
* Functional test for check cols snapshots using `updated_at`
* Comments to explain the test implementation
2022-04-21 06:56:19 -06:00
Emily Rockman
979e1c74bf
add new GHA for dependabot PRs ( #5065 )
...
* add new GHA for dependabot PRs
* Add automated changelog entry
* code cleanup
* remove changelog file
* permissions tweak
* Add automated changelog yaml from template
* update commit author
* Add automated changelog yaml from template
* fix formatting, remove changelog
* revert to separate files and comment out changelog check temporarily
* Add automated changelog yaml from template
* add back changelog check, update how commit works
* remove file
* Add automated changelog yaml from template
* WIP update to use PAT
* update PAT name
* remove file
* Add automated changelog yaml from template
* format file with quotes
* delete file
* Add automated changelog yaml from template
* remove extra line
* remove file
* Add automated changelog yaml from template
* Delete Dependencies-20220418-194629.yaml
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
2022-04-20 15:55:58 -05:00
Gerda Shank
7d0fccd63f
Convert docs_generate_tests to new framework ( #5058 )
2022-04-20 11:30:34 -04:00
Emily Rockman
37b8b65aad
cleanup changelog files on main after cutting release branch ( #5112 )
...
* cleanup cahngelog files on main after release branch was cut
* last file that is backported
2022-04-19 15:24:45 -05:00
Emily Rockman
0211668361
CT-476 convert deprecation tests ( #5034 )
...
* first pass at 012 test conversion
* convert 012_deprecation tests
* add logic around dropping schema
* swap exception
* added clarifying comment
2022-04-19 13:07:05 -05:00
leahwicz
f8c8322bb4
Updating backport action to latest ( #5082 )
...
* Updating backport action to latest
* Updating to PR trigger with permissions instead
This is a better model for closing down all permissions and just granting what we actually want
* Updating IF when merged and backport label exists
* Changing to only trigger on label being added
2022-04-18 15:36:43 -04:00
Mila Page
14c2bd9959
Ct 488/migrate simple seed ( #5060 )
...
* (finally) idiomatically rewrite a class of tests into the new framework.
* Get simple seed mostly working with design tweaks needed.
* Revamp tests to use more of the framework. Fix TODOs
* Complete migration of 005 and remove old files.
* Fix BOM test for Windows and add changelog entry
* Finalize tests in the adapter zone per conversation with Chenyu.
Co-authored-by: Mila Page <versusfacit@users.noreply.github.com >
2022-04-14 13:18:31 -07:00
Emily Rockman
8db6bac1db
move deprecation check outside package caching ( #5069 )
...
* move deprecation check outside package caching
* add changelog
2022-04-14 14:13:38 -05:00
Joel Labes
080dd41876
Clarify steps to reopen a stale issue ( #4802 )
...
I haven't added a message for stale PRs because they're likely to only impact the opening user (who I assume can reopen their own PR) and they're less of a problem. Happy to add that in as well, as well as to take feedback on the specific phrasing here.
2022-04-14 17:50:27 +02:00
Amy Byrum
8e9702cec5
Add updated dbt diagram for readme ( #5055 )
2022-04-13 14:34:17 -06:00
Michael Manganiello
5ff81c244e
Flexibilize MarkupSafe pinned version ( #5039 )
...
* Flexibilize MarkupSafe pinned version
The current `MarkupSafe` pinned version has been added in #4746 as a
temporary fix for #4745 .
However, the current restrictive approach isn't compatible with other
libraries that could require an even older version of `MarkupSafe`, like
Airflow `2.2.2` [0], which requires `markupsafe>=1.1.1, <2.0`.
To avoid that issue, we can allow a greater range of supported
`MarkupSafe` versions. Considering the direct dependency `dbt-core` has
is `Jinja2==2.11.3`, we can use its pinning as the lower bound, which is
`MarkupSafe>=0.23` [1].
This fix should be also backported this to `1.0.latest` for inclusion in
the next v1.0 patch.
[0] https://github.com/adamantike/airflow/blob/2.2.2/setup.cfg#L125
[1] https://github.com/pallets/jinja/blob/2.11.3/setup.py#L53
2022-04-13 13:44:27 -06:00
github-actions[bot]
cfe81e81fd
Bumping version to 1.2.0a1 ( #5045 )
...
* Bumping version to 1.2.0a1
* Fixing spacing issue
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
2022-04-12 17:00:07 -04:00
leahwicz
365414b5fc
Bumping manifest schema to v5 ( #5032 )
...
* Bumping manifest schema to v5
* Adding changelog
2022-04-12 16:06:24 -04:00
Nathaniel May
ec46be7368
Perf regression testing - overhaul of readme and runner ( #4602 )
2022-04-12 16:00:55 -04:00
Stu Kilgore
f23a403468
Update version output and logic ( #5029 )
...
Update version output and logic
2022-04-12 14:36:55 -05:00
Benoit Perigaud
15ad34e415
Add selected_resources to the Jinja context ( #5001 )
...
* Add selected_resources in the Jinja context
* Add tests for the Jinja variable selected_resources
* Add Changie entry for the addition of selected_resources
* Move variable to the ProviderContext
* Move selected_resources from ModelContext to ProviderContext
* Update unit tests for context to cater for the new selected_resources variable
* Move tests to a Class where tests are run after a dbt build
2022-04-12 10:25:45 -06:00
Jeremy Cohen
bacc891703
Add experimental cache_selected_only config ( #5036 )
...
* cache schema for selected models
* Create Features-20220316-003847.yaml
* rename flag, update postgres adapter
rename flag to cache_selected_only, update postgres adapter: function _relations_cache_for_schemas
* Update Features-20220316-003847.yaml
* added test for cache_selected_only flag
* formatted as per pre-commit
* Add breaking change note for adapter plugin maintainers
* Fix whitespace
* Add a test
Co-authored-by: karunpoudel-chr <poudel.karun@gmail.com >
Co-authored-by: karunpoudel-chr <62040859+karunpoudel@users.noreply.github.com >
2022-04-12 18:04:39 +02:00
Emily Rockman
a2e167761c
add more complete logic around changelog contributors section ( #5037 )
...
* add more complete logic around changelog contributors section
* add instructions for future core team members
* Update .changie.yaml
2022-04-12 10:35:21 -05:00
Emily Rockman
cce8fda06c
Add enabled as a source config ( #5008 )
...
* initial pass at source config test w/o overrides
* Update tests/functional/sources/test_source_configs.py
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
* Update tests/functional/sources/test_source_configs.py
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
* tweaks from feedback
* clean up some test logic - add override tests
* add new fields to source config class
* fix odd formatting
* got a test working
* removed unused tests
* removed extra fields from SourceConfig class
* fixed next failing unit test
* adding back missing import
* first pass at adding source table configs
* updated remaining tests to pass
* remove source override tests
* add comment for config merging
* changelog
* remove old comments
* hacky fix for permission test
* remove unhelpful test
* adding back test file that was accidentally deleted
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
Co-authored-by: Nathaniel May <nathaniel.may@fishtownanalytics.com >
Co-authored-by: Chenyu Li <chenyu.li@dbtlabs.com >
2022-04-12 10:27:29 -05:00
leahwicz
dd4ac1ba4a
Updating tests and doc to support Python 3.10 ( #5025 )
...
* Updating tests and doc to support Python 3.10
* Single quotes needed for python version matrix
* Adding changelog
2022-04-12 10:52:44 -04:00
Sung Won Chung
401ebc2768
Smart Source Freshness Runs ( #4256 )
...
* first draft
* working selector code
* remove debug print logs
* copy test template
* add todo
* smarter depends on graph searching notes
* add excluded source children nodes
* remove prints and clean up logger
* opinionated fresh node selection
* better if handling
* include logs with meaningul info
* add concurrent selectors note
* cleaner logging
* Revert "Merge branch 'main' of https://github.com/dbt-labs/dbt into feature/smart-source-freshness-runs"
This reverts commit 7fee4d44bf , reversing
changes made to 17c47ff42d .
* tidy up logs
* remove comments
* handle criterion that does not match nodes
* use a blank set instead
* Revert "Revert "Merge branch 'main' of https://github.com/dbt-labs/dbt into feature/smart-source-freshness-runs""
This reverts commit 71125167a1 .
* make compatible with rc and new logger
* new log format
* new selector flag name
* clarify that status needs to be correct
* compare current and previous state
* correct import
* add current state
* remove print
* add todo
* fix error conditions
* clearer refresh language
* don't run wasteful logs
* remove for now
* cleaner syntax
* turn generator into set
* remove print
* add fresh selector
* data bookmarks matter only
* remove exclusion logic for status
* keep it DRY
* remove unused import
* dynamic project root
* dynamic cwd
* add TODO
* simple profiles_dir import
* add default target path
* headless path utils
* draft work
* add previous sources artifact read
* make PreviousState aware of t-2 sources
* make SourceFreshSelectorMethod aware of t-2 sources
* add archive_path() for t-2 sources to freshness.py
* clean up merged branches
* add to changelog
* rename file
* remove files
* remove archive path logic
* add in currentstate and previousstate defaults
* working version of source fresher
* syntax source_fresher: works
* fix quoting
* working version of target_path default
* None default to sources_current
* updated source selection semantics
* remove todo
* move to test_sources folder
* copy over baseline source freshness tests
* clean up
* remove test file
* update state with version checks
* fix flake tests
* add changelog
* fix name
* add base test template
* delegate tests
* add basic test to ensure nothing runs
* add another basic test
* fix test with copy state
* run error test
* run warn test
* run pass test
* error handling for runtime error in source freshness
* error handling for runtime error in source freshness
* add back fresher selector condition
* top level selector condition
* add runtime error test
* testing source fresher test selection methods
* fix formatting issues
* fix broken tests
* remove old comments
* fix regressions in other tests
* add Anais test cases
* result selector test case
Co-authored-by: Matt Winkler <matt.winkler@fishtownanalytics.com >
2022-04-12 15:08:06 +02:00
Emily Rockman
83612a98b7
cache after retrying instead of while retrying ( #5028 )
2022-04-11 19:53:11 -05:00
Leopoldo Araujo
827eae2750
Added no-print flag ( #4854 )
...
* Added no-print flag
* Updated changelog
* Updated changelog
* Removed changes from CHANGELOG.md
* Updated CHANGELOG.MD with changie
* Update .changes/unreleased/Features-20220408-114118.yaml
Co-authored-by: Emily Rockman <ebuschang@gmail.com >
Co-authored-by: Emily Rockman <ebuschang@gmail.com >
2022-04-11 13:48:34 -05:00
Emily Rockman
3a3bedcd8e
Update index file for docs generation ( #4995 )
...
* Update index file for docs generation
* add changelog entries
2022-04-11 11:31:03 -05:00
Stu Kilgore
c1dfb4e6e6
Convert --version tests to pytest ( #5026 )
...
Convert --version tests to pytest
2022-04-11 11:04:45 -05:00
Gerda Shank
5852f17f0b
Fix hard_delete_snapshot test to do the right thing. ( #5020 )
2022-04-08 16:18:01 -04:00
dependabot[bot]
a94156703d
Bump black from 22.1.0 to 22.3.0 ( #4972 )
...
* Bump black from 22.1.0 to 22.3.0
2022-04-08 15:10:36 -05:00
Ian Knox
2b3fb7a5d0
updated docker readme CT-452 ( #5018 )
2022-04-08 14:30:25 -05:00
Ian Knox
5f2a43864f
Decouple project creation logic from tasks CT-299 ( #4981 )
2022-04-08 14:28:37 -05:00
Ian Knox
88fbc94db2
added git-blame-ignore-revs file ( #5019 )
2022-04-08 14:20:43 -05:00
Chenyu Li
6c277b5fe1
make graph_selection tests just checking selection ( #5012 )
...
* make graph_selection tests just checking selection
* use util function
2022-04-08 11:04:54 -06:00
Chenyu Li
40e64b238c
adapter_methods ( #4939 )
...
* adapter_methods
* fix fixture scope
* update table compare method
* remove unneeded part
* update test name and comment
2022-04-08 08:32:21 -06:00
Ian Knox
581bf51574
updated event message ( #5011 )
2022-04-08 09:12:49 -05:00
Gerda Shank
899b0ef224
Remove TableComparison and convert existing calls to use dbt.tests.util ( #4986 )
2022-04-07 13:04:03 -04:00
Matthew McKnight
3ade206e86
init push up of converted unique_key tests ( #4958 )
...
* init push up of converted unique_key tests
* testing cause of failure
* adding changelog entry
* moving non basic test up one directory to be more broadly part of adapter zone
* minor changes to the bad_unique_key tests
* removed unused fixture
* moving tests to base class and inheriting in a simple class
* taking in chenyu's changes to fixtures
* remove older test_unique_key tests
* removed commented out code
* uncommenting seed_count
* v2 based on feedback for base version of testing, plus small removal of leftover breakpoint
* create incremental test directory in adapter zone
* commenting out TableComparision and trying to implement check_relations_equal instead
* remove unused commented out code
* changing cast for date to fix test to work on bigquery
2022-04-07 11:29:52 -05:00
agoblet
58bd750007
add DO_NOT_TRACK environment variable support ( #5000 )
2022-04-07 11:45:29 -04:00
Matthew McKnight
0ec829a096
include directory README ( #4685 )
...
* start of a README for the include directory
* minor updates
* minor updates after comments from gerda and emily
* trailing space issue?
* black formatting
* minor word change
* typo update
* minor fixes and changelog creation
* remove changelog
2022-04-06 11:53:59 -05:00
Emily Rockman
7f953a6d48
[CT-352] catch and retry malformed json ( #4982 )
...
* catch None and malformed json reponses
* add json.dumps for format
* format
* Cache registry request results. Avoid one request per version
* updated to be direct in type checking
* add changelog entry
* add back logic for none check
* PR feedback: memoize > global
* add checks for expected types and keys
* consolidated cache and retry logic
* minor cleanup for clarity/consistency
* add pr review suggestions
* update unit test
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
2022-04-05 10:44:00 -05:00
Snyk bot
0b92f04683
[Snyk] Security upgrade python from 3.9.9-slim-bullseye to 3.10.3-slim-bullseye ( #4963 )
...
* fix: docker/Dockerfile to reduce vulnerabilities
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-DEBIAN11-EXPAT-2403512
- https://snyk.io/vuln/SNYK-DEBIAN11-EXPAT-2406127
- https://snyk.io/vuln/SNYK-DEBIAN11-OPENSSL-2388380
- https://snyk.io/vuln/SNYK-DEBIAN11-OPENSSL-2426309
- https://snyk.io/vuln/SNYK-DEBIAN11-OPENSSL-2426309
* add changelog entry
Co-authored-by: Nathaniel May <nathaniel.may@fishtownanalytics.com >
2022-04-04 12:57:43 -04:00
Jeremy Cohen
3f37a43a8c
Remove unneeded code in default snapshot materialization ( #4993 )
...
* Rm unneeded create_schema in snapshot mtlzn
* Add changelog entry
2022-04-04 17:25:53 +02:00
Gerda Shank
204d53516a
Create a dbt.tests.adapter release when releasing dbt and postgres ( #4948 )
...
* update black version for pre-commit
2022-03-29 19:38:33 -04:00
Jeremy Cohen
5071b00baa
Custom names for generic tests ( #4898 )
...
* Support user-supplied name for generic tests
* Support dict-style generic test spec
* Add changelog entry
* Add TODO comment
* Rework raise_duplicate_resource_name
* Add functional tests
* Update comments, rm TODO
* PR feedback
2022-03-25 17:09:35 +01:00
Emily Rockman
81118d904a
Convert source tests ( #4935 )
...
* convert 059 to new test framework
* remove replaced tests
* WIP, has pre-commit errors
* WIP, has pre-commit errors
* one failing test, most issued resolved
* fixed final test and cleaned up fixtures
* remove converted tests
* updated test to work on windows
* remove config version
2022-03-24 09:19:54 -05:00
Jeremy Cohen
69cdc4148e
Cosmetic changelog/changie fixups ( #4944 )
...
* Reorder kinds in changie
* Reorder change categories for v1.1.0b1
* Update language for breaking change
* Contributors deserve an h3
* Make pre-commit happy? Update language
* Rm trailing whitespace
2022-03-24 12:17:55 +01:00
Chenyu Li
1c71bf414d
remove capping version of typing extensions ( #4934 )
2022-03-23 14:08:26 -04:00
Chenyu Li
7cf57ae72d
add compliation and cache tracking ( #4912 )
2022-03-23 14:05:50 -04:00
kadero
1b6f95fef4
Fix inconsistent timestamps snapshots ( #4513 )
2022-03-23 12:05:42 -05:00
github-actions[bot]
38940eeeea
Bumping version to 1.1.0b1 ( #4933 )
...
* Bumping version to 1.1.0b1
2022-03-23 09:28:50 -05:00
Ian Knox
6c950bad7c
updated bumpversion ( #4932 )
2022-03-22 15:02:52 -05:00
Joel Labes
5e681929ae
Add space before justification periods ( #4744 )
...
* Update format.py
* Update CHANGELOG.md
* add change file
Co-authored-by: Gerda Shank <gerda@dbtlabs.com >
2022-03-22 15:18:38 -04:00
Matthew McKnight
ea5a9da71e
update of macro for postgres/redshift use of unique_key as a list ( #4858 )
...
* pre-commit additions
* added changie changelog entry
* moving integration test over
* Pair programming
* removing ref to mapping as seems to be unnecessary check, unique_key tests pass locally for postgres
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
2022-03-22 10:24:21 -05:00
leahwicz
9c5ee59e19
Updating backport workflow to use forked action ( #4920 )
2022-03-22 09:10:30 -04:00
Emily Rockman
55b1d3a191
changie - convert changelogs to yaml files and make quality of life improvements ( #4917 )
...
* convert changelog to changie yaml files
* update contributor format and README instructions
* update action to rerun when labeled/unlabled
* remove synchronize from action
* remove md file replaced by the yaml
* add synchronize and comment of what's happening
* tweak formatting
2022-03-21 20:15:52 -05:00
Ian Knox
a968aa7725
added permissions settings for docker release workflow ( #4903 )
2022-03-18 10:40:05 -05:00
Gerda Shank
5e0a765917
Set up adapter testing framework for use by adapter test repos ( #4846 )
2022-03-17 18:01:09 -04:00
Ian Knox
0aeb9976f4
remove missing setup.py file (holdover from pip install dbt ( #4896 )
2022-03-17 16:52:02 -05:00
Nathaniel May
30a7da8112
[HOTFIX] update dbt-extractor dependency ( #4890 )
...
* use pep 0440 compatible release operator for dbt-extractor dependency. bump to 0.4.1.
2022-03-17 16:44:30 -04:00
Matthew McKnight
f6a9dae422
FEAT: new columns in snapshots for adapters w/o bools ( #4871 )
...
* FEAT: new columns in snapshots for adapters w/o bools
* trigger gha workflow
* using changie to make changelog
* updating to be on par with main
Co-authored-by: swanderz <swanson.anders@gmail.com >
2022-03-17 10:10:23 -05:00
Gerda Shank
62a7163334
Use cli_vars instead of context to create package and selector renderers ( #4878 )
2022-03-17 09:27:39 -04:00
Mila Page
e2f0467f5d
Add bugged version tag value to finds. ( #4816 )
...
* Change property file version exception to reflect current name and offer clearer guidance in comments.
* Add example in case of noninteger version tag just to drive the point home to readers.
2022-03-16 14:59:48 -07:00
Mila Page
3e3ecb1c3f
get_response type hint is AdapterResponse only. ( #4869 )
...
* get_response type hint is AdapterResponse only.
* Propagate changes to get_response return type to execute
2022-03-16 14:54:39 -07:00
Nathaniel May
27511d807f
update test project ( #4875 )
2022-03-16 16:35:07 -04:00
Ian Knox
15077d087c
python 3.10 support ( #4866 )
...
* python 3.10 support
2022-03-15 19:35:28 -05:00
Emily Rockman
5b01cc0c22
catch all requests exceptions to retry ( #4865 )
...
* catch all requests exceptions to retry
* add changelog
2022-03-15 11:57:07 -05:00
Chenyu Li
d1bcff865d
pytest conversion test_selection, schema_tests, fail_fast, permission ( #4826 )
2022-03-15 11:12:30 -04:00
Emily Rockman
0fce63665c
Small changie fixes ( #4857 )
...
* fix broken links, update GHA to not repost comment
* tweak GHA
* convert GHA used
* consolidate GHA
* fix PR numbers and pull comment as var
* fix name of workflow step
* changie merge to fix link at top of changelog
* add changelog yaml
2022-03-11 14:54:33 -06:00
Emily Rockman
1183e85eb4
Er/ct 303 004 simple snapshot ( #4838 )
...
* convert single test in 004
* WIP
* incremental conversion
* WIP test not running
* WIP
* convert test_missing_strategy, cross_schema_snapshot
* comment
* converting to class based test
* clean up
* WIP
* converted 2 more tests
* convert hard delete test
* fixing inconsistencies, adding comments
* more conversion
* implementing class scope changes
* clean up unsed code
* remove old test, get all new ones running
* fix typos
* append file names with snapshot to reduce collision
* moved all fixtures into test files
* stop using tests as fixtures
2022-03-11 14:52:54 -06:00
dependabot[bot]
3b86243f04
Update typing-extensions requirement from <3.11,>=3.7.4 to >=3.7.4,<4.2 in /core ( #4719 )
...
* Update typing-extensions requirement in /core
Updates the requirements on [typing-extensions](https://github.com/python/typing ) to permit the latest version.
- [Release notes](https://github.com/python/typing/releases )
- [Changelog](https://github.com/python/typing/blob/master/typing_extensions/CHANGELOG )
- [Commits](https://github.com/python/typing/compare/3.7.4...4.1.0 )
---
updated-dependencies:
- dependency-name: typing-extensions
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com >
* Empty-Commit
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: ChenyuLi <chenyu.li@dbtlabs.com >
2022-03-10 15:42:20 -05:00
willbowditch
c251dae75e
[CT-271] [Feature] not_null test selects column instead of * ( #4777 )
...
* Only select target column for not_null test
* If storing failures include all columns in the select, if not, only select the column being tested
It's desirable for this test to include the full row output when using --store-failures. If the query result stored in the database contained just the null values of the null column, it can't do much to contextualize why those rows are null.
* Update changelog
* chore: update changelog using changie
* Revert "Update changelog"
This reverts commit 281d805959 .
2022-03-09 21:31:15 -05:00
Emily Rockman
ecfd77f1ca
Small updates to clarify change destinations ( #4841 )
...
* update to reflect this branch is for the 1.1 release
* update to use next
* remove next logic
* add yaml changes also marked for unreleased 1.0.4
2022-03-08 13:18:24 -06:00
Emily Rockman
9a0abc1bfc
Automate changelog ( #4743 )
...
* initial setup to use changie
* added `dbt-core` to version line
* fix formatting
* rename to be more accurate
* remove extra file
* add stug for contributing section
* updated docs for contributing and changelog
* first pass at changelog check
* Fix workflow name
* comment on handling failure
* add automatic contributors section via footer
* removed unused initialization
* add script to automate entire changelog creation and handle prereleases
* stub out README
* add changelog entry!
* no longer need to add contributors ourselves
* fixed formatted and excluded core team
* fix typo and collapse if statement
* updated to reflect automatic pre-release handling
Removed custom script in favor of built in pre-release functionality in new version of changie.
* update contributing doc
* pass at GHA
* fix path
* all changed files
* more GHA work
* continued GHA work
* try another approach
* testing
* adding comment via GHA
* added uses for GHA
* more debugging
* fixed formatting
* another comment attempt
* remove read permission
* add label check
* fix quotes
* checking label logic
* test forcing failure
* remove extra script tag
* removed logic for having changelog
* Revert "removed logic for having changelog"
This reverts commit 490bda8256 .
* remove unused workflow section
* update header and readme
* update with current version of changelog
* add step failure for missing changelog file
* fix typos and formatting
* small tweaks per feedback
* Update so changelog end up onlywith current version, not past
* update changelog to recent contents
* added the rest of our releases to previous release list
* clarifying the readme
* updated to reflect current changelog state
* updated so only 1.1 changes are on main
2022-03-07 20:12:33 -06:00
Gerda Shank
490d68e076
Switch to using class scope fixtures ( #4835 )
...
* Switch to using class scope fixtures
* Reorganize some graph selection tests because of ci errors
2022-03-07 14:38:36 -05:00
Stu Kilgore
c45147fe6d
Fix macro modified from previous state ( #4820 )
...
* Fix macro modified from previous state
Previously, if the first node selected by state:modified had multiple
dependencies, the first of which had not been changed, the rest of the
macro dependencies of the node would not be checked for changes. This
commit fixes this behavior, so the remainder of the macro dependencies
of the node will be checked as well.
2022-03-07 08:23:59 -06:00
Gerda Shank
bc3468e649
Convert tests in dbt-adapter-tests to use new pytest framework ( #4815 )
...
* Convert tests in dbt-adapter-tests to use new pytest framework
* Filter out ResourceWarning for log file
* Move run_sql to dbt.tests.util, fix check_cols definition
* Convert jaffle_shop fixture and test to use classes
* Tweak run_sql methods, rename some adapter file pieces, add comment
to dbt.tests.adapter.
* Add some more comments
2022-03-03 16:53:41 -05:00
Kyle Wigley
8fff6729a2
simplify and cleanup gha workflow ( #4803 )
2022-03-02 10:21:39 -05:00
varun-dc
08f50acb9e
Fix stdout piped colored output on MacOS and Linux ( #4792 )
...
* Fix stdout pipe output coloring
* Update CHANGELOG.md
Co-authored-by: Chenyu Li <chenyulee777@gmail.com >
Co-authored-by: Chenyu Li <chenyulee777@gmail.com >
2022-03-01 17:23:51 -05:00
Chenyu Li
436a5f5cd4
add coverage ( #4791 )
2022-02-28 09:17:33 -05:00
Emily Rockman
aca710048f
ct-237 test conversion 002_varchar_widening_tests ( #4795 )
...
* convert 002 integration test
* remove original test
* moved varchar test under basic folder
2022-02-25 14:25:22 -06:00
Emily Rockman
673ad50e21
updated index file to fix DAG errors for operations & work around null columns ( #4763 )
...
* updated index file to fix DAG errors for operations
* update index file to reflect dbt-docs fixes
* add changelog
2022-02-25 13:02:26 -06:00
Chenyu Li
8ee86a61a0
rewrite graph selection ( #4783 )
...
* rewrite graph selection
2022-02-25 12:09:11 -05:00
Gerda Shank
0dda0a90cf
Fix errors on Windows tests in new tests/functional ( #4767 )
...
* [#4781 ] Convert reads and writes in project fixture to text/utf-8 encoding
* Switch to using write_file and read_file functions
* Add comment
2022-02-25 11:13:15 -05:00
Gerda Shank
220d8b888c
Fix "dbt found two resources" error with multiple snapshot blocks in one file ( #4773 )
...
* Fix handling of multiple snapshot blocks in partial parsing
* Update tests for partial parsing snapshots
2022-02-25 10:54:07 -05:00
dependabot[bot]
42d5812577
Bump black from 21.12b0 to 22.1.0 ( #4718 )
...
Bumps [black](https://github.com/psf/black ) from 21.12b0 to 22.1.0.
- [Release notes](https://github.com/psf/black/releases )
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md )
- [Commits](https://github.com/psf/black/commits/22.1.0 )
---
updated-dependencies:
- dependency-name: black
dependency-type: direct:development
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-24 13:28:23 -05:00
Ian Knox
dea4f5f8ff
update flake8 to remove line length req ( #4779 )
2022-02-24 11:22:25 -06:00
Dmytro Kazanzhy
8f50eee330
Fixed misspellings, typos, and duplicated words ( #4545 )
2022-02-22 18:05:43 -05:00
Gerda Shank
8fd8dfcf74
Initial pass at switching integration tests to pytest ( #4691 )
...
Author: Emily Rockman <emily.rockman@dbtlabs.com >
route logs to dbt-core/logs instead of each test folder (#4711 )
* Initial pass at switching integration tests to pytest
* Reorganize dbt.tests.tables. Cleanup adapter handling
* Move run_sql to TestProjInfo and TableComparison.
Add comments, cleanup adapter schema setup
* Tweak unique_schema name generation
* Update CHANGELOG.md
2022-02-22 15:34:14 -05:00
Hein Bekker
10b27b9633
Deduplicate postgres relations ( #3058 ) ( #4521 )
...
* Deduplicate postgres relations (#3058 )
* Add changelog entry for #3058 , #4521
2022-02-21 16:48:15 -06:00
Gerda Shank
5808ee6dd7
Fix bug accessing target in deps and clean commands ( #4758 )
...
* Create DictDefaultNone for to_target_dict in deps and clean commands
* Update test case to handle
* update CHANGELOG.md
* Switch to DictDefaultEmptyStr for to_target_dict
2022-02-21 13:26:29 -05:00
Jeremy Cohen
a66fe7f467
Pin MarkupSafe==2.0.1 ( #4746 )
2022-02-18 14:35:27 +01:00
Gerda Shank
18fef38702
Ensure meta is both at node top level and in node.config. Fix snapshots with schema config. ( #4726 )
...
* Do not overwrite node.meta with empty patch.meta
* Restore config_call_dict in snapshot node transform
* Test for snapshot with schema file config
* Test for meta in both toplevel node and node config
2022-02-17 12:15:11 -05:00
Ian Knox
3ad61d5d81
ignore markdown files for trim-trailing-whitespace hook ( #4727 )
2022-02-16 10:25:52 -06:00
Emily Rockman
bb1f5b43be
Initital pass at deps README ( #4686 )
...
* Initital pass at README
* Finished the sentence
* fixed typo and added changelog
2022-02-15 13:58:22 -06:00
Michiel De Smet
a642b20abc
Allow override of Column string and numeric type by classes inheritin… ( #4604 )
...
* Allow override of Column string and numeric type by classes inheriting from the Column class
* updating based on new black formatter
Co-authored-by: Matthew McKnight <91097623+McKnight-42@users.noreply.github.com >
Co-authored-by: Matthew McKnight <matthew.mcknight@dbtlabs.com >
2022-02-14 15:22:43 -06:00
Ian Knox
c112050455
Pre commit Hooks (black, flake8, mypy, etc) [CT-105] ( #4639 )
...
Added pre-commit and configured hooks (black, flake8, mypy, white space formatters)
Removed code checks from tox
updated CI
2022-02-11 12:57:16 -06:00
Ian Knox
43e3fc22c4
Reformat core [CT-104 CT-105] ( #4697 )
...
Reformatting dbt-core via black, flake8, mypy, and assorted pre-commit hooks.
2022-02-11 12:17:31 -06:00
elizabeth martens
41c6177ae2
Add --quiet flag and print Jinja function ( #4701 )
...
* Add `--quiet` flag
* Add print() macro
* Update tests for --quiet and print()
* Updating changelog
* Apply suggestions from PR review
2022-02-10 13:24:42 -06:00
Tristan Willy
72ecd1ce74
task init: support older click v7.0 ( #4681 )
...
* task init: support older click v7.0
`dbt init` uses click for interactively setting up a project. The
version constraints currently ask for click >= 8 but v7.0 has nearly the
same prompt/confirm/echo API. prompt added a feature that isn't used.
confirm has a behavior change if the default is None, but
confirm(..., default=None) is not used. Long story short, we can relax
the version constraint to allow installing with an older click library.
Ref: Issue #4566
* Update CHANGELOG.md
Co-authored-by: Chenyu Li <chenyulee777@gmail.com >
Co-authored-by: Chenyu Li <chenyulee777@gmail.com >
2022-02-07 14:14:22 -05:00
Nathaniel May
2d0b975b6c
fix test to use a secret username ( #4682 )
2022-02-04 14:57:03 -05:00
Rachel
8a0bc39a66
Set flags from args in lib module for dbt-server ( #4623 )
2022-02-04 10:14:41 -05:00
nkyuray
f3c7b6bfd1
adapter compability messaging added. ( #4565 )
...
* adapter compability messaging added.
* edited plugin version compatibility message
* edited test version for plugin compability
* compare using only major and minor
* Add checking PYPI and update changelog
Co-authored-by: Chenyu Li <chenyulee777@gmail.com >
Co-authored-by: ChenyuLi <chenyu.li@dbtlabs.com >
2022-02-03 17:27:31 -05:00
Nathaniel May
0391e4e53a
add changelog entry for #4665 ( #4673 )
2022-02-03 15:48:05 -05:00
Gerda Shank
3ad3c21886
[ #2479 ] Allow unique_id to take a list ( #4618 )
...
* Add unique_key to NodeConfig
`unique_key` can be a string or a list.
* merge.sql update to work with unique_key as list
extend the functionality to support both single and multiple keys
Signed-off-by: triedandtested-dev (Bryan Dunkley) <bryan@triedandtested.dev >
* Updated test to include unique_key
Signed-off-by: triedandtested-dev (Bryan Dunkley) <bryan@triedandtested.dev >
* updated tests
Signed-off-by: triedandtested-dev (Bryan Dunkley) <bryan@triedandtested.dev >
* Fix unit and integration tests
* Update Changelog for 2479/4618
Co-authored-by: triedandtested-dev (Bryan Dunkley) <bryan@triedandtested.dev >
2022-02-03 12:55:06 -05:00
Nathaniel May
6e0ed751e1
Avoid saving secrets in SecretContext ( #4665 )
2022-02-03 12:54:45 -05:00
Gerda Shank
c43c79a995
Initial file creation of code documentation READMEs ( #4654 )
2022-02-02 18:29:47 -05:00
Ian Knox
d6cc8b3042
Docker release CT-3 ( #4616 )
...
* new docker setup
* formatting
* Updated spark: support for extras
* Added third-party adapter support
* More selective lib installs for spark
* added docker to bumpversion
* Updated refs to be tag-based because bumpversion doesn't understand 'latest'
* Updated docs per PR feedback
* reducing RUNs and formatting/pip best practices changes
* Added multi-architecture support and small test script, updated docs
* typo
* Added a few more tests
* fixed tests output, clarified dbt-postgres special case-ness
* Fix merge conflicts
* formatting
* Updated spark: support for extras
* Added third-party adapter support
* More selective lib installs for spark
* added docker to bumpversion
* Updated refs to be tag-based because bumpversion doesn't understand 'latest'
* Updated docs per PR feedback
* reducing RUNs and formatting/pip best practices changes
* Added multi-architecture support and small test script, updated docs
* typo
* Added a few more tests
* fixed tests output, clarified dbt-postgres special case-ness
* changelog
* basic framework
* PR ready excepts docs
* PR feedback
2022-02-01 16:49:33 -06:00
Chenyu Li
2f4a6e33ec
fix changelog for a community pr ( #4659 )
2022-02-01 13:50:58 -05:00
Alec Wang
b9867e89cb
added semver official regex pattern ( #4644 )
...
* added semver official regex pattern
* removed extra character
* added comma
* added contribution
* fixed contribution
* Update CHANGELOG.md
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
2022-02-01 08:19:37 -05:00
Nathaniel May
13b18654f0
Guard against unnecessarily calling dump_graph in logging ( #4619 )
...
* add lazy type and apply to cache events
2022-01-31 14:14:34 -05:00
Jeremy Cohen
aafa1c7f47
Change InvalidRefInTestNode level to DEBUG ( #4647 )
...
* Debug-level test depends on disabled
* Add PR link to Changelog
2022-01-31 18:28:43 +01:00
Jeremy Cohen
638e3ad299
Drop support for Python <3.7.2 ( #4643 )
...
* Drop support for 3.7.1 + 3.7.2
* Rm root level setup.py
* Rm 'dbt' pkg from build-dist script
* Fixup changelog
2022-01-31 17:31:20 +01:00
Emily Rockman
d9cfeb1ea3
Retry after failure to download or failure to open files ( #4609 )
...
* add retry logic, tests when extracting tarfile fails
* fixed bug with not catching empty responses
* specify compression type
* WIP test
* more testing work
* fixed up unit test
* add changelog
* Add more comments!
* clarify why we do the json() check for None
2022-01-31 10:26:51 -06:00
Chenyu Li
e6786a2bc3
fix comparision for new model/body ( #4631 )
...
* fix comparison for new model/body
2022-01-31 10:33:35 -05:00
leahwicz
13571435a3
Initial addition of CODEOWNERS file ( #4620 )
...
* Initial addition of CODEOWNERS file
* Proposed sub-team ownership (#4632 )
* Updating for the events module to be both language and execution
* Adding more comment details
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
2022-01-27 16:23:55 -05:00
Gerda Shank
efb890db2d
[ #4504 ] Use mashumaro for serializing logging events ( #4505 )
2022-01-27 14:43:26 -05:00
Niall Woodward
f3735187a6
Run check_if_can_write_profile before create_profile_using_project_profile_template [CT-67] [Backport 1.0.latest] ( #4447 )
...
* Run check_if_can_write_profile before create_profile_using_project_profile_template
* Changelog
Co-authored-by: Ian Knox <81931810+iknox-fa@users.noreply.github.com >
2022-01-27 11:17:28 -06:00
Gerda Shank
3032594b26
[ #4554 ] Don't require a profile for dbt deps and clean commands ( #4610 )
2022-01-25 12:26:44 -05:00
Joel Labes
1df7a029b4
Clarify "incompatible package version" error msg ( #4587 )
...
* Clarify "incompatible package version" error msg
* Clarify error message when they shouldn't fall fwd
2022-01-24 18:33:45 -05:00
leahwicz
f467fba151
Changing Jira mirroring workflows to point to shared Actions ( #4615 )
2022-01-24 12:20:12 -05:00
Amir Kadivar
8791313ec5
Validate project names in interactive dbt init ( #4536 )
...
* Validate project names in interactive dbt init
- workflow: ask the user to provide a valid project name until they do.
- new integration tests
- supported scenarios:
- dbt init
- dbt init -s
- dbt init [name]
- dbt init [name] -s
* Update Changelog.md
* Add full URLs to CHANGELOG.md
Co-authored-by: Chenyu Li <chenyulee777@gmail.com >
Co-authored-by: Chenyu Li <chenyulee777@gmail.com >
2022-01-21 18:24:26 -05:00
leahwicz
7798f932a0
Add Backport Action ( #4605 )
2022-01-21 12:40:55 -05:00
Nathaniel May
a588607ec6
drop support for Python 3.7.0 and 3.7.1 ( #4585 )
2022-01-19 12:24:37 -05:00
Joel Labes
348764d99d
Rename data directory to seeds ( #4589 )
...
* Rename data directory to seeds
* Update CHANGELOG.md
2022-01-19 10:04:35 -06:00
Gerda Shank
5aeb088a73
[ #3988 ] Fix test deprecation warnings ( #4556 )
2022-01-12 17:03:11 -05:00
leahwicz
e943b9fc84
Mirror labels to Jira ( #4550 )
...
* Adding Jira label mirroring
* Fixing bad step name
2022-01-05 09:29:52 -05:00
leahwicz
892426eecb
Mirroring issues to Jira ( #4548 )
...
* Adding issue creation Jira Action
* Adding issue closing Jira Action
* Add labeling logic
2022-01-04 17:00:03 -05:00
Emily Rockman
1d25b2b046
test name standardization ( #4509 )
...
* rename tests for standardization
* more renaming
* rename tests to remove duplicate numbers
* removed unused file
* removed unused files in 016
* removed unused files in 017
* fixed schema number mismatch 027
* fixed to be actual directory name 025
* remove unused dir 029
* remove unused files 039
* remove unused files 053
* updated changelog
2022-01-04 11:36:47 -06:00
github-actions[bot]
da70840be8
Bumping version to 1.0.1 ( #4543 )
...
* Bumping version to 1.0.1
* Update CHANGELOG.md
* Update CHANGELOG.md
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
2022-01-03 13:04:50 -05:00
leahwicz
7632782ecd
Removing Docker from bumpversion script ( #4542 )
2022-01-03 12:48:03 -05:00
Nathaniel May
6fae647097
copy over windows compat logic for colored log output ( #4474 )
2022-01-03 12:37:36 -05:00
leahwicz
fc8b8c11d5
Commenting our Docker portion of Version Bump ( #4541 )
2022-01-03 12:37:20 -05:00
Topherhindman
26a7922a34
Fix small typo in architecture doc ( #4533 )
2022-01-03 12:00:04 +01:00
Emily Rockman
c18b4f1f1a
removed unused code in unit tests ( #4496 )
...
* removed unused code
* add changelog
* moved changelog entry
2021-12-23 08:26:22 -06:00
Nathaniel May
fa31a67499
Add Structured Logging ADR ( #4308 )
2021-12-22 10:26:14 -05:00
Ian Knox
742cd990ee
New Dockerfile ( #4487 )
...
New Dockerfile supporting individual db adapters and architectures
2021-12-22 08:29:21 -06:00
Gerda Shank
8463af35c3
[ #4523 ] Fix error with env_var in hook ( #4524 )
2021-12-20 14:19:05 -05:00
github-actions[bot]
b34a4ab493
Bumping version to 1.0.1rc1 ( #4517 )
...
* Bumping version to 1.0.1rc1
* Update CHANGELOG.md
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
2021-12-19 15:33:38 -05:00
Jeremy Cohen
417ccdc3b4
Fix bool coercion to 0/1 ( #4512 )
...
* Fix bool coercion
* Fix unit test
2021-12-19 10:30:25 -05:00
Emily Rockman
7c46b784ef
scrub message of secrets ( #4507 )
...
* scrub message of secrets
* update changelog
* use new scrubbing and scrub more places using git
* fixed small miss of string conv and missing raise
* fix bug with cloning error
* resolving message issues
* better, more specific scrubbing
2021-12-17 16:05:57 -06:00
Gerda Shank
067b861d30
Improve checking of schema version for pre-1.0.0 manifests ( #4497 )
...
* [#4470 ] Improve checking of schema version for pre-1.0.0 manifests
* Check exception code instead of message in test
2021-12-16 13:30:52 -05:00
Emily Rockman
9f6ed3cec3
update log message to use adapter name ( #4501 )
...
* update log message to use adapter name
* add changelog
2021-12-16 11:46:28 -06:00
Nathaniel May
43edc887f9
Simplify Log Destinations ( #4483 )
2021-12-16 11:40:05 -05:00
Emily Rockman
6d4c64a436
compile new index file for docs ( #4484 )
...
* compile new index file for docs
* Add changelog
* move changleog entries for docs changes
2021-12-16 10:09:02 -06:00
Gerda Shank
0ed14fa236
[ #4464 ] Check specifically for generic node type for some partial parsing actions ( #4465 )
...
* [#4464 ] Check specifically for generic node type for some partial parsing actions
* Add check for existence of macro file in saved_files
* Check for existence of patch file in saved_files
2021-12-14 16:28:40 -05:00
Emily Rockman
51f2daf4b0
updated DepsStartPackageInstall event to use package name ( #4482 )
...
* updated event to user package name
* add changelog
2021-12-14 14:25:29 -06:00
Matthew McKnight
76f7bf9900
made change to test of str ( #4463 )
...
* made change to test of str
* changelog update
2021-12-13 11:55:19 -06:00
Matthew McKnight
3fc715f066
updating contributing.md based on suggestions from updates to adapter… ( #4356 )
...
* updating contributing.md based on suggestions from updates to adapter contributing files.
* removed section refering to non-postgres databases for core contributing.md
* making suggested changes to contributing.md based on kyle's initial lookover
* Update CONTRIBUTING.md
Co-authored-by: Kyle Wigley <kyle@fishtownanalytics.com >
Co-authored-by: Kyle Wigley <kyle@fishtownanalytics.com >
2021-12-10 13:14:49 -06:00
Rebekka Moyson
b6811da84f
Fix dbt docs overview to working url ( #4442 )
...
* Fix to working url
* add fix to changelog
2021-12-08 10:30:41 -06:00
Nathaniel May
1dffccd9da
point latest version check to dbt-core package ( #4434 )
2021-12-03 16:13:38 -05:00
github-actions[bot]
9ed9936c84
Bumping version to 1.0.0 ( #4431 )
...
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
2021-12-03 13:27:46 -05:00
Jeremy Cohen
e75ae8c754
Changelog entries for rc3 -> final ( #4389 )
...
* Changelog entries for rc3 -> final
* More updates
* Final entry
* Last fix, and the date
* These few, these happy few
2021-12-03 19:16:46 +01:00
Nathaniel May
b68535b8cb
relax version specifier for dbt-extractor ( #4427 )
2021-12-03 12:56:03 -05:00
Nathaniel May
5310498647
add new interop tests for black-box json log schema testing ( #4327 )
2021-12-03 12:51:28 -05:00
Ian Knox
22b1a09aa2
stringify generic exceptions ( #4424 )
2021-12-03 10:32:22 -06:00
Jeremy Cohen
6855fe06a7
Info vs debug text formatting ( #4418 )
2021-12-03 14:36:42 +01:00
Jeremy Cohen
affd8619c2
Sources aren't materialized ( #4417 )
2021-12-03 14:36:35 +01:00
Jeremy Cohen
b67d5f396b
Add flag to main.py. Reinstantiate after flags ( #4416 )
2021-12-03 14:36:25 +01:00
Emily Rockman
b3039fdc76
add node type codes to more events + more hook log data ( #4378 )
...
* add node type codes to more events + more hook log
* minor fixes
* renames started/finished keys
* made process more clear
* fixed errors
* Put back report_node_data in fresshness.py
Co-authored-by: Gerda Shank <gerda@dbtlabs.com >
2021-12-02 19:25:57 -05:00
Nathaniel May
9bdf5fe74a
use reference keys instead of relations ( #4410 )
2021-12-02 18:35:51 -05:00
Emily Rockman
c675c2d318
Logging README ( #4395 )
...
* WIP
* more README cleanup
* readme tweaks
* small tweaks
* wording updates
2021-12-02 17:04:23 -06:00
Ian Knox
2cd1f7d98e
user configurable event buffer size ( #4411 )
2021-12-02 16:47:31 -06:00
Jeremy Cohen
ce9ac8ea10
Rollover + backup for dbt.log ( #4405 )
2021-12-02 22:10:11 +01:00
Jeremy Cohen
b90ab74975
A few final logging touch-ups ( #4388 )
...
* Rm unused events, per #4104
* More structured ConcurrencyLine
* Replace \n prefixes with EmptyLine
* Reimplement ui.warning_tag to centralize logic
* Use warning_tag for deprecations too
* Rm more unused event types
* Exclude EmptyLine from json logs
* loglines are not always created by events (#4406 )
Co-authored-by: Nathaniel May <nathaniel.may@fishtownanalytics.com >
2021-12-02 22:09:46 +01:00
Emily Rockman
6d3c3f1995
update file name ( #4402 )
2021-12-02 15:04:29 -06:00
Nathaniel May
74fbaa18cd
change json override strategy ( #4396 )
2021-12-02 15:04:52 -05:00
Emily Rockman
fc7c073691
allow log_format to be set in profile configs ( #4394 )
2021-12-02 13:51:45 -06:00
leahwicz
29f504e201
Fix release process ( #4385 )
2021-12-02 11:18:49 -05:00
Nathaniel May
eeb490ed15
use rfc3339 format for log time stamps ( #4384 )
2021-12-02 09:44:10 -05:00
Gerda Shank
c220b1e42c
[ #4354 ] Different output for console and file logs ( #4379 )
...
* [#4354 ] Different output for console and file logs
* Tweak some log formats
* Change loging of thread names
2021-12-02 08:23:25 -05:00
Jeremy Cohen
d973ae9ec6
Tiny touchups for deps, clean ( #4366 )
...
* Use actual profile name for log msg
* Raise clean dep warning iff configured path missing
2021-12-02 12:12:49 +01:00
Ian Knox
f461683df5
Add windows OS error supressing for temp dir cleanups ( #4380 )
2021-12-01 17:25:56 -06:00
Nathaniel May
41ed976941
move event code up a level ( #4381 )
...
move event code up a level plus minor fixes
2021-12-01 17:30:19 -05:00
Gerda Shank
e93ad5f118
Make the stdout logger actually go to stdout ( #4368 )
2021-11-30 17:48:23 -05:00
Emily Rockman
d75ed964f8
only log events in cache.py when flag is set set ( #4369 )
...
flag is --log-cache-events
2021-11-30 15:17:08 -06:00
Nathaniel May
284ac9b138
better dataclass field handling ( #4361 )
...
fix serializing dataclass fields so they show up at all
2021-11-30 13:34:57 -05:00
github-actions[bot]
7448ec5adb
Bumping version to 1.0.0rc3 ( #4363 )
...
* Bumping version to 1.0.0rc3
* Updating Changelog for release
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
2021-11-30 09:35:03 -05:00
Emily Rockman
caa6269bc7
add node_info to relevant logs ( #4336 )
...
* WIP
* fixed some merg issues
* WIP
* first pass with node_status logging
* add node details to one more
* another pass at node info
* fixed failures
* convert to classes
* more tweaks to basic implementation
* added in ststus, organized a bit
* saving broken state
* working state with lots of todos
* formatting
* add start/end tiemstamps
* adding node_status logging to more events
* adding node_status to more events
* Add RunningStatus and set in node
* Add NodeCompiling and NodeExecuting events, switch to _event_status dict
* add _event_status to SourceDefinition
* small tweaks to NodeInfo
* fixed misnamed attr
* small fix to validation
* rename logging timestamps to minimize name collision
* fixed flake failure
* move str formatting to events
* incorporate serialization changes
* add node_status to event_to_serializable_dict
* convert nodeInfo to dict with dataclass builtin
* Try to fix failing unit, flake8, mypy tests (#4362 )
* fixed leftover merge conflict
Co-authored-by: Gerda Shank <gerda@dbtlabs.com >
2021-11-30 09:34:28 -05:00
Gerda Shank
31691c3b88
Events with graph_func include actual output of graph_func ( #4360 )
2021-11-29 20:20:22 -05:00
Ian Knox
3a904a811f
Event buffer for structlog ( #4358 )
...
Add Internal event buffer
Co-authored-by: Nathaniel May <nathaniel.may@fishtownanalytics.com >
2021-11-29 20:12:20 -05:00
Nathaniel May
b927a31a53
make json serialization overridable for events ( #4326 )
...
* simplify scrubbing
* add overridable serialize method to events
* add imperfect test for json serialization of events
Co-authored-by: Ian Knox <ian.knox@fishtownanalytics.com >
Co-authored-by: Kyle Wigley <kyle@fishtownanalytics.com >
2021-11-29 18:19:34 -05:00
Kyle Wigley
d8dd75320c
set invocation id when generating psuedo config ( #4359 )
2021-11-29 17:29:12 -05:00
Nathaniel May
a613556246
add thread_name to json output ( #4353 )
2021-11-29 14:01:50 -05:00
Jeremy Cohen
8d2351d541
Logging: restore previous (small) behaviors ( #4341 )
...
* Log formatting from flags earlier
* WARN-level stdout for list task
* Readd tracking events to File
* PR feedback, annotate hacks
* Revert "PR feedback, annotate hacks"
This reverts commit 5508fa230b .
* This is maybe better
* Annotate main.py
* One more comment in base.py
* Update changelog
2021-11-29 19:05:39 +01:00
leahwicz
f72b603196
Adding release workflow ( #4288 )
2021-11-29 10:37:14 -05:00
Gerda Shank
4eb17b57fb
Provide function to set the invocation_id ( #4351 )
2021-11-29 10:15:19 -05:00
Cor
85a4b87267
Use cls in classmethod ( #4345 )
...
Instead of calling the class explicitly, use the `cls` variable instead.
2021-11-29 09:57:52 -05:00
jan zens
0d320c58da
fix typo in UnparsedSourceDefinition.__post_serialize_ ( #4349 )
...
* fix typo in UnparsedSourceDefinition.__post_serialize_
fix typo in UnparsedSourceDefinition.__post_serialize_
* update CHANGELOG.md
update CHANGELOG.md
add #4349
* Update changelog
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
2021-11-29 11:36:11 +01:00
Emilie Lima Schario
ed1ff2caac
Adjust logic when finding approx matches for model or test matching ( #4076 )
...
* adjust logic when finding approx matches
* update changelog
* Update core/dbt/adapters/base/relation.py
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
* Update changelog
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
2021-11-29 11:20:01 +01:00
sarah-weatherbee
d80646c258
adds additional augmented assignment statements ( #4315 ) ( #4331 )
...
* adds additional augmented assignment statements (#4315 )
* Per PR comments, revised CHANGELOG.md to note change and contributor info
2021-11-27 09:04:40 -06:00
Matthew McKnight
a9b4316346
Mc knight 42/test event codes ( #4338 )
...
* pushing up to get eye on from Nate
* updating to compare
* latest push
* finished test for duplicate codes with a lot of help from Nate
* resolving suggestions
* removed duplicated code in types.py, made minor changes to test_events.py
* added missing func call
2021-11-24 16:03:43 -06:00
Gerda Shank
36776b96e7
[ #4337 ] Always create an invocation_id, even when not tracking ( #4340 )
2021-11-24 16:54:17 -05:00
Jeremy Cohen
7f2d3cd24f
Fix static parser tracking logic ( #4332 )
...
* Fix static parser tracking logic
* Add changelog note
2021-11-24 17:26:56 +01:00
Gerda Shank
d046ae0606
[ #4253 ] Support partial parsing of env_vars in metrics definitions ( #4322 )
2021-11-23 15:02:47 -05:00
Gerda Shank
e8c267275e
[ #4254 ] Change some CompilationExceptions to ParsingException in the parser ( #4328 )
2021-11-23 13:50:00 -05:00
github-actions[bot]
a4951749a8
Bumping version to 1.0.0rc2 ( #4321 )
...
* Bumping version to 1.0.0rc2
* Update changelog
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
2021-11-22 21:26:15 +01:00
Ian Knox
e1a2e8d9f5
Add codes to all log events (re-work of PR #4268 ) ( #4319 )
...
* re-work of old branch
2021-11-22 13:14:33 -06:00
Emily Rockman
f80c78e3a5
add logic to scrub more than str types ( #4317 )
2021-11-22 12:58:10 -06:00
Emily Rockman
c541eca592
structured logging: add data attributes to json log output ( #4301 )
...
* simplified data construction
* fixed missed scrubbing of secrets
* switched to vars()
* scrub entire log line, update how attributes get pulled
* get ahead of serialization errors
* store if data is serialized and modify values instead of a copy of values
* fixed unused import from merge
2021-11-19 15:43:26 -06:00
Nathaniel May
726aba0586
version logging ( #4289 )
...
* start adding version logging, noticed some wrong stuff
* fix bad pid and ts
* remove level format on json logs
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com >
2021-11-19 14:53:50 -06:00
Jeremy Cohen
d300addee1
SecretContext for secret env vars, profiles + packages only ( #4311 )
...
* SecretContext for secret env vars
* Cleanup exception. Add + edit tests
* Add changelog entry
2021-11-19 19:36:19 +01:00
Kyle Wigley
d5d16f01f4
Fix flags import ( #4307 )
2021-11-18 14:59:49 -05:00
Kyle Wigley
2cb26e2699
Add supported dbt tasks ( #4200 )
2021-11-18 14:05:00 -05:00
Nathaniel May
b4793b4f9b
Fix adapter failures due to string formatting issues ( #4305 )
...
fix adapter failures due to string formatting issues
2021-11-18 12:54:20 -05:00
Gerda Shank
045e70ccf1
[ #4298 ] Fix 'created_at' in ParsedMetric to allow recalculating metrics depends_on refs ( #4299 )
2021-11-18 09:29:09 -05:00
Jeremy Cohen
ba23395c8e
Fix metrics count in compile stats ( #4292 )
...
* Fix metrics count in compile stats
* Add changelog entry
2021-11-18 09:28:13 +01:00
Joel Labes
0aacd99168
Get prerelease packages when specifically requested ( #4295 )
...
* Get prerelease packages when specifically required. Add test validating it works
* Update CHANGELOG.md
2021-11-18 09:11:49 +01:00
Nathaniel May
e4b5d73dc4
adjust level length for text only ( #4303 )
...
adjust level length for text log lines only
2021-11-17 17:32:15 -05:00
Gerda Shank
bd950f687a
[ #4252 ] Serialization error when missing quotes in metrics model ref() call ( #4287 )
2021-11-17 17:14:32 -05:00
Gerda Shank
aea23a488b
[ #4272 ] Move validator keyword argument in jinja 'config.get' to after 'default' ( #4297 )
2021-11-17 17:12:26 -05:00
Jeremy Cohen
22731df07b
Fix: default log formatting ( #4302 )
...
* Respect log formatting
* PR feedback
2021-11-17 21:10:14 +01:00
Jeremy Cohen
c55be164e6
Separate warnings. Fix duplication ( #4291 )
2021-11-17 18:01:28 +01:00
kadero
9d73304c1a
Alow snapshot defer ( #4296 )
...
* Alow snapshot defer
* Update changelog
2021-11-17 16:56:37 +01:00
Nathaniel May
719b2026ab
Minor Cleanup of Structured Logging Module ( #4266 )
...
* cleanup structured logging module
* update adapter logger to preserve new-style logging formatting
2021-11-16 20:22:11 -05:00
kadero
22416980d1
Avoid errors when missing column in yaml doc ( #4285 )
...
* Update postgres__alter_column_comment
* Update changelog
* Add integration test
2021-11-16 13:22:18 +01:00
Gerda Shank
3d28b6704c
[ #3689 ] Fix filesystem searcher and tests that mock it ( #4271 )
2021-11-15 09:46:17 -05:00
Mila Page
5d1b104e1f
Feature/3997 profiles test selection flag ( #4270 )
...
* Address 3997. Test selection flag can be in profile.yml.
* Per Jerco's 4104 PR unresolved comments, unify i.s. predicate and add env var.
* Couple of flake8 touchups.
* Classier error handling using enum semantics.
* Cherry-pick in part of Gerda's commit to hopefully avoid a future merge conflict.
* Add 3997 to changelog.
Co-authored-by: Mila Page <versusfacit@users.noreply.github.com >
2021-11-15 14:07:22 +01:00
Jeremy Cohen
4a8a68049d
Try removing dupe logging during integration tests ( #4275 )
2021-11-15 11:00:29 +01:00
Jeremy Cohen
4b7fd1d46a
Update dbt-postgres readme ( #4263 )
...
* Update dbt-postgres readme
* Rm redshift references
* Update plugins/postgres/README.md
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
* Update plugins/postgres/README.md
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
2021-11-12 17:12:00 +01:00
github-actions[bot]
0722922c03
Bumping version to 1.0.0rc1 ( #4234 )
...
* Bumping version to 1.0.0rc1
* Update changelog
* Add Dockerfile to bumpversion, update reqs
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
2021-11-10 14:24:40 +01:00
kadero
40321d7966
Dbt init with provided project name ( #4249 )
...
* Dbt init with provided project name
* Update changelog.md
* Fix changelog.md
* Fix typo in help
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
2021-11-10 11:58:49 +01:00
Nathaniel May
434f3d2678
Merge pull request #4055 from dbt-labs/feature/structured-logging
...
Add Structured Logging
2021-11-09 17:42:19 -05:00
Jeremy Cohen
6dd9c2c5ba
Env var shim to enable legacy logger ( #4255 )
...
* Env var shim to reenable logbook
* Rename to ENABLE_LEGACY_LOGGER
2021-11-09 23:04:47 +01:00
Nathaniel May
5e6be1660e
configure event logger for integration tests ( #4257 )
...
* apply test fixes
* remove presto test
2021-11-09 16:13:13 -05:00
Nathaniel May
31acb95d7a
rebased on main and added new partial parsing event
2021-11-09 11:40:18 -05:00
Nathaniel May
683190b711
fixes
2021-11-09 11:26:01 -05:00
Nathaniel May
ebb84c404f
postgres adapter to use new logger
2021-11-09 11:26:01 -05:00
Nathaniel May
2ca6ce688b
whitespace change
2021-11-09 11:26:01 -05:00
Nathaniel May
a40550b89d
std logger for structured logging ( #4231 )
...
structured logging powered by the stdlib logger
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com >
Co-authored-by: Ian Knox <81931810+iknox-fa@users.noreply.github.com >
2021-11-09 11:26:01 -05:00
Ian Knox
b2aea11cdb
Struct log for adapter call sites ( #4189 )
...
graph call sites for structured logging
Co-authored-by: Nathaniel May <nathaniel.may@fishtownanalytics.com >
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com >
2021-11-09 11:26:01 -05:00
Emily Rockman
43b39fd1aa
removed redundant timestamp ( #4239 )
2021-11-09 11:26:01 -05:00
Emily Rockman
5cc8626e96
updates associated with merging main
...
- removed 3 new log call sites and replaced with structured logs
- removed 2 unused struc logs
2021-11-09 11:26:01 -05:00
Nathaniel May
f95e9efbc0
use event types in main even before the logger is set up. ( #4219 )
2021-11-09 11:26:01 -05:00
Nathaniel May
25c974af8c
lazy logging in event module ( #4210 )
...
* switches on debug level to guard against expensive messages
* adds memoization to msg construction
2021-11-09 11:26:01 -05:00
Emily Rockman
b5c6f09a9e
remove unused import ( #4217 )
2021-11-09 11:26:01 -05:00
Emily Rockman
bd3e623240
test/integration call sites ( #4209 )
...
* added struct logging to base
* fixed merge wierdness
* convert to use single type for integration tests
* converted to 3 reusable test types in sep module
* tweak message
* clean up and making test_types complete for future
* fix missed import
2021-11-09 11:26:01 -05:00
Emily Rockman
63343653a9
trivial logger removal ( #4216 )
2021-11-09 11:26:01 -05:00
Emily Rockman
d8b97c1077
call sites in core/dbt (excluding main.py) ( #4202 )
...
* add struct logging to compilation
* add struct logging to tracking
* add struct logging to utils
* add struct logging to exceptions
* fixed some misc errors
* updated to send raw ex, removed resulting circ dep
2021-11-09 11:26:01 -05:00
Emily Rockman
e0b0edaeed
deps call sites ( #4199 )
...
* add struct logging to base
* add struct logging to git
* add struct logging to deps
* remove blank line
* fixed stray merge error
2021-11-09 11:26:01 -05:00
Emily Rockman
3cafc9e13f
task callsites: part 2 ( #4188 )
...
* add struct logging to docs serve
* remove merge fluff
* struct logging to seed command
* converting print to use structured logging
* more structured logging print conversion
* pulling apart formatting more
* added struct logging by disecting printer.py
* add struct logging to runnable
* add struct logging to task init
* fixed formatting
* more formatting and moving things around
2021-11-09 11:26:01 -05:00
Nathaniel May
13f31aed90
scrub the secrets ( #4203 )
...
scrub secrets in event module
2021-11-09 11:26:01 -05:00
Nathaniel May
d513491046
Show Exception should trigger a stack trace ( #4190 )
2021-11-09 11:26:01 -05:00
Emily Rockman
281d2491a5
task call sites part 1 ( #4183 )
...
* add struct logging to base.py
* struct logging in run_operation
* add struct logging to base
* add struct logging to clean
* add struct logging to debug
* add struct logging to deps
* fix errors
* add struct logging to run.py
* fixed flake error
* add struct logging to geneerate
* added debug level stack trace
* fixed flake error
* added struct logging to compile
* added struct logging to freshness
* cleaned up errors
* resolved bug that broke everything
* removed accidental import
* fixed bug with unused args
2021-11-09 11:26:01 -05:00
Emily Rockman
9857e1dd83
parser call sites ( #4177 )
...
* convert generic_test to structured logging
* convert macros to structured logging
* add struc logging to most of manifest.py
* add struct logging to models.py
* added struct logging to partial.py
* finished conversion of manifest
* fixing errors
* fixed 1 todo and added another
* fixed bugs from merge
2021-11-09 11:26:01 -05:00
Emily Rockman
6b36b18029
config call sites ( #4169 )
...
* update config use structured logging
* WIP
* minor cleanup
* fixed merge error
* added in ShowException
* added todo to remove defaults after dropping 3.6
* removed todo that is obsolete
2021-11-09 11:26:01 -05:00
Ian Knox
d8868c5197
Dataclass compatibility ( #4180 )
...
* use __post_init__() instead of fake dataclass member vars
2021-11-09 11:26:01 -05:00
Emily Rockman
b141620125
contracts call sites ( #4166 )
...
* first pass adding structured logging
2021-11-09 11:26:01 -05:00
Emily Rockman
51d8440dd4
Change Graph logger call sites ( #4165 )
...
graph call sites for structured logging
2021-11-09 11:26:01 -05:00
Nathaniel May
5b2562a919
Client call sites ( #4163 )
...
update log call sites with new event system
2021-11-09 11:26:01 -05:00
Nathaniel May
44a9da621e
Handle exec info ( #4168 )
...
handle exec info
2021-11-09 11:26:01 -05:00
Emily Rockman
69aa6bf964
context call sites ( #4164 )
...
* updated context dir to new structured logging
2021-11-09 11:26:01 -05:00
Nathaniel May
f9ef9da110
Initial structured logging work with fire_event ( #4137 )
...
add event type modeling and fire_event calls
2021-11-09 11:26:01 -05:00
Nathaniel May
57ae9180c2
init
2021-11-09 11:26:01 -05:00
Jeremy Cohen
efe926d20c
Change user instead of pass ( #4250 )
2021-11-09 13:10:34 +01:00
Jeremy Cohen
1081b8e720
Improve error msg on pip install dbt ( #4244 )
2021-11-09 10:40:45 +01:00
Kyle Wigley
8205921c4b
Update docs ( #4241 )
...
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
2021-11-08 19:47:28 -05:00
Jeremy Cohen
da6c211611
Wrap get_batch_size() in return() ( #4240 )
2021-11-09 00:46:15 +01:00
Jeremy Cohen
354c1e0d4d
Rm py36 tests, pkg metadata, bump reqs ( #4223 )
2021-11-09 00:19:09 +01:00
Gerda Shank
855419d698
[ #4071 ] Add metrics feature ( #4235 )
...
* first cut at supporting metrics definitions
* teach dbt about metrics
* wip
* support partial parsing for metrics
* working on tests
* Fix some tests
* Add partial parsing metrics test
* Fix some more tests
* Update CHANGELOG.md
* Fix partial parsing yaml file to correct model syntax
Co-authored-by: Drew Banin <drew@fishtownanalytics.com >
2021-11-08 17:44:01 -05:00
Gerda Shank
e94fd61b24
Issue message instead of exception when patch does not have a matching ( #4236 ) node
2021-11-08 15:35:14 -05:00
Kyle Wigley
4cf9b73c3d
Raise parsing error instead of compilation when extracting test args ( #4237 )
2021-11-08 14:51:52 -05:00
Jeremy Cohen
8442fb66a5
Reorganize global project (macros) ( #4154 )
...
* Add integration tests
* Reorganize + dispatch more global macros
* Reorg materializations subdir
* Move around + document generic tests
* Fix failing tests
* Fix merge conflict
* Grab fix from #4148
* PR feedback
* Fixup
* Add load_relation back, it was nice
* Last few test fixes
* Rm incremental_upsert, now unused
* Add changelog entry
2021-11-08 19:09:54 +01:00
dependabot[bot]
f8cefa3eff
Update agate requirement from <1.6.2,>=1.6 to >=1.6,<1.6.4 in /core ( #3585 )
...
Updates the requirements on [agate](https://github.com/wireservice/agate ) to permit the latest version.
- [Release notes](https://github.com/wireservice/agate/releases )
- [Changelog](https://github.com/wireservice/agate/blob/master/CHANGELOG.rst )
- [Commits](https://github.com/wireservice/agate/compare/1.6.0...1.6.3 )
---
updated-dependencies:
- dependency-name: agate
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-08 18:41:04 +01:00
dependabot[bot]
d83e0fb8d8
Bump mashumaro from 2.5 to 2.9 in /core ( #4193 )
...
Bumps [mashumaro](https://github.com/Fatal1ty/mashumaro ) from 2.5 to 2.9.
- [Release notes](https://github.com/Fatal1ty/mashumaro/releases )
- [Commits](https://github.com/Fatal1ty/mashumaro/compare/v2.5...v2.9 )
---
updated-dependencies:
- dependency-name: mashumaro
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-08 18:40:25 +01:00
Gerda Shank
3e9da06365
[ #3885 ] Skip partial parsing if project env vars change ( #4212 )
...
* [#3885 ] Skip partial parsing if project env vars change
* Support env_vars in the profile
2021-11-08 11:51:38 -05:00
Gerda Shank
bda70c988e
[ #3885 ] Partially parse when environment variables in schema files change ( #4162 )
...
* [#3885 ] Partially parse when environment variables in schema files
change
* Add documentation for test kwargs
* Add test and fix for schema configs with env_var
2021-11-08 11:28:43 -05:00
Rachel
229e897070
Clears adapters before registering to fix dbt-server cacheing behavior ( #4218 )
2021-11-08 10:33:39 -05:00
Benoit Perigaud
f20e83a32b
Fix/dbt deps retry none answer ( #4225 )
...
* Fix issue #4178
Allow retries when the answer is None
* Include fix for #4178
Allow retries when the answer from dbt deps is None
* Add link to the PR
* Update exception and shorten line size
* Add test when dbt deps returns None
2021-11-08 12:30:38 +01:00
Jeremy Cohen
dd84f9a896
Raise error on pip install dbt ( #4133 )
...
* Raise error on pip install dbt
* Fix relative path logic
* Do not build dist for dbt
* Fix long descriptions
* Trigger code checks
* Using root readme more trouble than good
* only fail on install, not build
* Edit dist script. Avoid README duplication
* jk, be less clever
* Ignore 'dbt' source distribution when testing
* Add changelog entry
Co-authored-by: Kyle Wigley <kyle@dbtlabs.com >
2021-11-07 17:55:30 +01:00
Mila Page
e6df4266f6
Parser no longer takes greedy. Accepts indirect selection, a bool. ( #4104 )
...
* Parser no longer takes greedy. Accepts indirect selection, a bool.
* Remove references to greedy and supporting functions.
* 1. Set testing flag default to True. 2. Improve arg parsing.
* Update tests and add new case for when flag unset.
* Update names and styling to fit test requirements. Add default value for option.
* Correct several failing tests now that default behavior was flipped.
* Tests expect eager on by default.
* All but selector test passing.
* Get integration tests working, add them, and mix in selector syntax.
* Clean code and correct test.
* Add changelog entry
Co-authored-by: Mila Page <versusfacit@users.noreply.github.com >
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
2021-11-07 17:41:56 +01:00
Christophe Oudar
b591e1a2b7
Use common columns for incremental schema changes ( #4170 )
...
* Use common columns for incremental schema changes
* on_schema_change:append_new_columns should gracefully handle column removal
* review changes
* Lean approach for `process_schema_changes` to simplify
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
2021-11-07 17:31:30 +01:00
Jeremy Cohen
3dab058c73
incorporate_indirect_nodes should pass if not needed (#4214 )
...
* Pass incorporate_indirect_nodes if not needed
* Fix flake8
* Add changelog entry
2021-11-05 16:55:58 +01:00
Robert
c7bc6eb812
Add error surfacing for git cloning errors ( #4124 )
...
* Add error surfacing for git cloning errors
* Update CHANGELOG.md
* Fix formatting and remove redundant except: raise
* Turn error handling for duplicate packages back on
2021-11-05 10:12:07 +01:00
Jeremy Cohen
c690ecc1fd
Fixup changelog ( #4206 )
2021-11-04 13:54:23 +01:00
Jeremy Cohen
73e272f06e
Add get_where_subquery to test namespace ( #4197 )
...
* Add get_where_subquery to test namespace
* Add integration test
* Fix test, add comment, smarter approach
* Fix unit tests
* Add changelog entry
2021-11-04 11:53:28 +01:00
leahwicz
95d087b51b
Bumping artifact versions for v1 ( #4191 )
...
* Bumping artifact versions for v1
* Adding schema in Changelog
* Update CHANGELOG.md
* Update changelog entry
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
2021-11-04 11:19:36 +01:00
Jeremy Cohen
40ae6b6bc8
Any subset, strict or not ( #4160 )
2021-11-02 17:59:46 +01:00
Jeremy Cohen
fe20534a98
Add extra graph edges for build only ( #4143 )
...
* Resolve extra graph edges for build only
* Fix flake8
* Change test to reflect functional change
* Rename method + args. Add changelog entry
2021-11-02 17:41:14 +01:00
leahwicz
dd7af477ac
Perf improvement to subgraph selection ( #4155 )
...
Perf improvement to get_subset_graph
Co-authored-by: Ian Knox <ian.knox@fishtownanalytics.com >
2021-10-29 16:06:09 -05:00
Jeremy Cohen
178f74b753
Fix comma if only removing columns in on_schema_change: sync_all_columns ( #4148 )
...
* Fix comma if only removing in on_schema_change: sync
* Add changelog entry
2021-10-28 10:19:05 +02:00
Emily Rockman
a14f563ec8
port error scrub from 0.21.latests up for main ( #4145 )
2021-10-27 14:06:22 -05:00
Kyle Wigley
ff109e1806
Expose lib to to run tasks and compile/execute sql ( #4111 )
2021-10-27 13:30:46 -04:00
Frank Cash
5e46694b68
assertRaisesRegexp => assertRaisesRegex ( #4136 )
...
* assertRaisesRegexp => assertRaisesRegex
* Update CHANGELOG.md
* Update CHANGELOG.md
2021-10-27 13:05:24 +02:00
Gerda Shank
73af9a56e5
[ #3885 ] Handle env_vars in partial parsing of SQL files ( #4101 )
...
* [#3885 ] Handle env_vars in partial parsing
* Comment method to build env_vars_to_source_files
2021-10-26 11:16:36 -04:00
kadero
d2aa920275
Feature: nullable error_after in source ( #3955 )
...
* Add nullable error after feature
* add merge_error_after method
* Fix FreshnessThreshold merged test
* Fix other tests
* Fix merge error after
* Fix test docs generate integration test
* Fix source integration test
* Typo and fix linting.
* Fix mypy test
* More terse way to express merge_freshness_time_thresholds
* Update Changelog.md
* Add integration test
* Fix conflict
* Fix contributing.md
* Fix integration tests
* Move up changelog entry
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
2021-10-26 15:23:57 +02:00
Gerda Shank
c34f3530c8
Use platform agnostic code when searching generic test directory ( #4131 )
2021-10-25 22:46:36 -04:00
Gerda Shank
c019a94206
[ #4128 ] Use platform agnostic way to check for tests/generic directory ( #4130 )
2021-10-25 13:46:49 -04:00
leahwicz
f9bdfa050b
Update CHANGELOG.md ( #4127 )
2021-10-25 13:44:47 -04:00
Gerda Shank
1b35d1aa21
Create more specific tests to debug partial parsing generic tests test ( #4129 )
2021-10-25 12:20:30 -04:00
Niall Woodward
420ef9cc7b
Add _fixed key config back for _choice options in profile_template.yml ( #4121 )
2021-10-22 17:52:54 +02:00
github-actions[bot]
02fdc2cb9f
Bumping version to 1.0.0b2 ( #4119 )
...
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
2021-10-22 11:42:19 -04:00
Joel Labes
f82745fb0c
Update git url in setup.py to be dbt-core ( #4059 )
...
Tripped over this while following links in #3968
2021-10-22 11:28:24 -04:00
Niall Woodward
3397bdc6a5
dbt init profile_template.yml improvements ( #4118 )
...
* Update profile_template.yml to use same syntax as target_options.yml
* Rename target_options to profile_template
* Update profile_template config spec
2021-10-22 17:18:58 +02:00
leahwicz
96e858ac0b
Add contributor to changelog ( #4120 )
...
* Add contributor to changelog
* Adding another denpendency
2021-10-22 10:45:08 -04:00
dependabot[bot]
f6a98b5674
Update packaging requirement from ~=20.9 to >=20.9,<22.0 in /core ( #3532 )
...
Updates the requirements on [packaging](https://github.com/pypa/packaging ) to permit the latest version.
- [Release notes](https://github.com/pypa/packaging/releases )
- [Changelog](https://github.com/pypa/packaging/blob/main/CHANGELOG.rst )
- [Commits](https://github.com/pypa/packaging/compare/20.9...21.0 )
---
updated-dependencies:
- dependency-name: packaging
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-22 10:40:10 -04:00
Joseph H
824f0bf2c0
Upgrade to python3-pip ( #4079 )
2021-10-22 10:28:29 -04:00
Cor
5648b1c622
Add project name to default search packages ( #4114 )
...
* Add project name to default search packages
We prefer macros in the project over the ones in the namespace (package)
* Add change to change log
* Use project_name instead of project
* Raise compilation error if no macros are found
* Update change log line
* Add test for package macro override
* Add JCZuurmond to contributor
* Fix typos
* Add test that should not over ride the package
* Add doc string to tests
2021-10-22 15:15:31 +02:00
Cor
bb1382e576
Allow to overwrite the default test selection ( #4116 )
...
It was not possible to overwrite the default test selection for the tox
integration env
2021-10-22 14:57:38 +02:00
Cor
085ea9181f
Remove redundant $() ( #4115 )
...
This executes the command (on mac OSX at least) thus creating an error
2021-10-22 14:56:11 +02:00
Emily Rockman
eace5b77a7
added test support for databases without boolean types ( #4091 )
...
* added support for dbs without boolean types
* catch errors a bit better
* moved changelog entry
* fixed tests and updated exception
* cleaned up bool check
* added positive test, removed self ref
2021-10-21 14:01:41 -05:00
Nathaniel May
1c61bb18e6
Fix static parser tracking again ( #4109 )
...
actually fix static parser tracking
2021-10-21 14:35:08 -04:00
Emily Rockman
f79a968a09
add generic tests to test-paths ( #4052 )
...
* removed overlooked breakpoint
* first pass
* save progress - singualr tests broken
* fixed to work with both generic and singular tests
* fixed formatting
* added a comment
* change to use /generic subfolder
* fix formatting issues
* fixed bug on code consolidation
* fixed typo
* added test for generic tests
* added changelog entry
* added logic to treat generic tests like macro tests
* add generic test to macro_edges
* fixed generic tests to match unique_ids
* fixed test
2021-10-21 11:42:23 -05:00
Niall Woodward
34c23fe650
Fix init command Windows integration tests ( #4107 )
...
* empty
* Trigger tests
* Use os.path.join for filepaths in test cases
2021-10-21 12:08:19 +02:00
Razvan Vacaru
3ae9475655
Fix setup_db.sh by waiting for pg_isready success return. Fixes #3876 ( #3908 )
...
* Fix setup_db.sh by waiting for pg_isready success return. Fixes #3876
* restored noaccess and dbtMixedCase creation and updated changelog and contributing md files
* restored root auth commands
* restored creation of dbt schema, aparently this is needed even if docker compose also creates it...
* pr comments: avoid infinite loop and quote variables
* Update changelog
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
2021-10-21 09:16:28 +02:00
Niall Woodward
11436fed45
dbt init Interactive profile creation (#3625 )
...
* Initial
* Further dev
* Make mypy happy
* Further dev
* Existing tests passing
* Functioning integration test
* Passing integration test
* Integration tests
* Add changelog entry
* Add integration test for init outside of project
* Fall back to target_options.yml when invalid profile_template.yml is provided
* Use built-in yaml with exception of in init
* Remove oyaml and fix tests
* Update dbt_project.yml in test comparison
* Create the profiles directory if it doesn't exist
* Use safe_load
* Update integration test
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
2021-10-20 18:38:49 +02:00
Nathaniel May
21a7b71657
Fix tracking bug for jinja sampling ( #4048 )
...
fix jinja sampling for static parser
2021-10-20 12:38:16 -04:00
leahwicz
280e9ad9c9
Removing release templates ( #4094 )
2021-10-19 09:04:59 -04:00
sungchun12
97f31c88e1
Smart Reruns from Failures/Errors ( #4017 )
...
* Add result: selection method
* make a copy for modified state test suite
* test case notes
* remove macro tests
* add a test setup command
* copy run results state
* passing test case, todos, split work
* clean up result:success test case
* start with build command and remove previous state where needed
* add error result selector tests for seed
* add another error seed test case
* remove todo
* passing build result:error tests
* single failure build test
* add passing test
* fix node assertions for tests
* fix tests
* draft fail+ tests
* add severity to test
* result:warn passing test
* result:warn+ passing tests
* add passing concurrent selector test
* add downstream flag
* add comment
* passing test
* fix test for dynamic node selection
* add build concurrent selector passing test
* add run test cases
* add integration tests for dbt test
* fix formatting
* rename test
* remove extra comments
* add extra newline
* add concurrent selector test / build cases
* clean up todos
* test all nodes
* DRY rebuild code
* test all nodes
* add TODO update assertion code
* cleaner assert code
* fix this test to have a fixed set
* more cleanup
* add changelog
* update concurrent selectors on dbt test
* remove todo
* Update changelog
* Apply suggestions from code review
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
* fix changelog
* fix Contributors headers
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
Co-authored-by: Matt Winkler <matt.winkler@fishtownanalytics.com >
2021-10-18 16:43:14 +02:00
Joseph H
5f483a6b13
Include only relational nodes in database_schema_set ( #4077 )
...
* Include only relational nodes in `database_schema_set`
* Update changelog entry
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
2021-10-18 12:30:00 +02:00
Jeremy Cohen
86f24e13db
Try removing connection_for from render_with_context ( #4062 )
...
* We don't need connections at parse time, actually?
* Add back import
* Update changelog
2021-10-18 11:53:31 +02:00
Jeremy Cohen
4bda8c8880
Enable test project hooks, flags.WHICH ( #4004 )
...
* Turn on project hooks test task
* Add flags.WHICH
* Rm unused env vars (RPC)
* Add changelog entry
2021-10-18 10:50:19 +02:00
Gerda Shank
80a5d27127
[ #4069 ] Use more standard disabled lookup functions ( #4073 )
2021-10-15 15:37:48 -04:00
Emily Rockman
4307a82058
fixed error_id typo and added associated tests ( #4070 )
...
* fixed error_id typo and added tests
* added changelog entry
2021-10-15 13:22:43 -05:00
Gerda Shank
5c01c42308
Add files in partial parsing test inadvertently not checked in ( #4072 )
2021-10-15 14:22:28 -04:00
Gerda Shank
80ba71682b
[ #4060 ] Refactor partial parsing test to avoid file collisions ( #4068 )
2021-10-15 13:54:43 -04:00
Kyle Wigley
26625e9627
ignore epic and discussion issues during stale check ( #4061 )
2021-10-14 10:46:09 -04:00
Jeremy Cohen
134e8423b7
Make unique_field optional, add docstring ( #4028 )
2021-10-14 16:27:21 +02:00
leahwicz
04a9195297
Schema check testing ( #3870 )
...
* Create schema-check.yml
* Adding PR trigger to test
* Adding branch name
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update schema-check.yml
* Update .github/workflows/schema-check.yml
Co-authored-by: Kamil Breguła <mik-laj@users.noreply.github.com >
* Revert "Update .github/workflows/schema-check.yml"
This reverts commit a2f1fa81ef .
Reverting b/c this broke something in the workflow
Co-authored-by: Kamil Breguła <mik-laj@users.noreply.github.com >
2021-10-14 10:12:51 -04:00
Kyle Wigley
8a10a69f59
update links to point to new repo name ( #4053 )
2021-10-13 20:28:14 -04:00
Gerda Shank
fd7c95d1d2
[ #4012 ] Performance: Use child_map to find tests for nodes in resolve_graph ( #4022 )
2021-10-13 17:03:41 -04:00
Kyle Wigley
79aa136301
pass --user arg ( #4046 )
2021-10-12 14:01:35 -04:00
Jeremy Cohen
3b5cec6cc6
Use dbt Core logo ( #4045 )
...
* Switch to dbt Core logo
* Absolute URL in readme
* Update changelog, too
* Make it bigger!
2021-10-12 18:29:28 +02:00
Emily Rockman
0e9a67956d
Er/1607 source paths ( #4008 )
...
* WIP to replace source_path with model_path
* updated some test to point to new testing branches
https://github.com/dbt-labs/dbt-integration-project needs updates to get all tests working
* deprecate souce_paths but not remove fully
* added deprecation test for path deprecation
* replace data-pathswith seed-paths: ['seeds']
ypdated tests to use default directory of 'seeds' instead of 'data'
* added test for exception when paths incorectly defined
source-paths and data-paths have been deprecated in favor of model-paths and seed-paths. You can still use the deprecated keys but you cannot define both the deprecated and new keys since we wouldn't know how to handle it.
* fixed test naming issue
* fix formatting issues, standardize names
* updated branches for dbt-integration-project
* updated changelog
* synced up rpc deletion messed up when merging
* changelog updates
2021-10-12 08:40:43 -05:00
Kyle Wigley
f9f0eab0b7
Add stale workflow ( #4034 )
...
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
2021-10-12 08:54:09 -04:00
Kyle Wigley
ed01b439cf
Update issue/PR tempates ( #4031 )
2021-10-12 08:53:10 -04:00
leahwicz
a398ed1a3e
Removing old Docker requirements files ( #4041 )
2021-10-12 08:48:58 -04:00
leahwicz
a818e6551b
Updating Changelog release date for v1.0.0.b1 ( #4040 )
2021-10-12 08:44:47 -04:00
github-actions[bot]
0a7471ebdc
Bumping version to 1.0.0b1 ( #4037 )
...
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
2021-10-11 12:54:06 -04:00
leahwicz
1a5bc83598
Fixing the version bump conditional statements ( #4024 )
2021-10-11 10:10:34 -04:00
Kyle Wigley
6e2df00648
Rip out rpc server code ( #3755 )
...
* rm rpc specfic code, still more references to rpc to clean up (rpc_method, integration tests, etc.)
* rm move references to rpc
* rm tests against rpc server
* rm more rpc files
* more code!
* sorry!
2021-10-08 11:43:55 -04:00
Jeremy Cohen
b338dfc99a
Bump version to 1.0.0a1 ( #4027 )
2021-10-08 08:36:59 -07:00
Jesse Lax
47033c459f
replace logger.warning with warn_or_error ( #4019 )
...
Uses `warn_or_error` to log _or_ raise errors depending on arguments
passed.
2021-10-08 07:51:08 -07:00
leahwicz
92a0930634
Creating version bump action ( #4015 )
...
* Creating version bump action
* Adding workflow dispatch inputs
2021-10-07 22:10:47 -04:00
Gerda Shank
cad1a48eb0
[ #3996 ] Fix or avoid several partial parsing bugs ( #4020 )
2021-10-07 11:51:42 -04:00
Nathaniel May
b451f87e3c
remove rpc tests ( #4002 )
2021-10-07 11:39:51 -04:00
Yu Ishikawa
20756290bc
Change the data type of sources of ParsedNodeDefaults ( #3967 )
...
* Change the data type of `sources` of `ParsedNodeDefaults`
* Add the statement about the change to `CHANGELOG.md`
* Add myself to the contributors
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-10-07 08:14:37 -07:00
Gerda Shank
f44c6ed136
[ #4013 ] Fix multiple disabled models ( #4018 )
2021-10-07 09:40:14 -04:00
Nathaniel May
b501f4317c
add jinja sampling to stable static parser ( #3970 )
2021-10-06 09:48:39 -04:00
Sam Lader
91b43f71bb
[ #3961 ] Enable cataloging of unlogged Postgres tables ( #3993 )
...
* Update catalog macro & add tests
* [#3961 ] Enable cataloging of unlogged Postgres tables
* Update contributors and add new lines to test
2021-10-06 06:48:03 -07:00
Emily Rockman
6fc64f0d3b
Make default project paths, init project consistent ( #4007 )
...
Update the default project paths to be `analysis-paths = ['analyses']` and `test-paths = ['tests]`. Also have starter project set `analysis-paths: ['analyses']` from now on. Fixed all associated tests.
2021-10-06 07:02:35 -05:00
Emily Rockman
ee5c697645
Rename default installation directory for packages ( #3999 )
...
* Change the default dbt packages installation directory to `dbt_packages` from `dbt_modules`. Also rename `module-path` to `packages-install-path` to allow default overrides of package install directory. Deprecation warning added for projects using the old `dbt_modules` name without specifying a `packages-install-path`.
* fixed deprecation test bug
* Fixed wording on deprecation warning.
2021-10-05 13:54:13 -05:00
Emily Rockman
3caec08ccb
enacted deprecation for dispatch-packages, cleaned up deprecations te… ( #3975 )
...
* enacted deprecation for dispatch-packages, cleaned up deprecations tests for unused macros/models. still need to clean up unused code.
* more work to catch packages use
* fixed tests for removing packages on adapter.dispatch.
* cleaned out folder for 012_deprecation_tests to remove unused models/data/macros
* removed obsolete code due to patching for packages arg in adapter.dispatch
* updated exception name
* added deprecation change to changelog.
2021-10-05 09:05:23 -05:00
leahwicz
f7680379fc
Porting back Changelog changes ( #4001 )
2021-10-05 09:29:51 -04:00
Jeremy Cohen
3789acc5a7
Move redshift, snowflake, bigquery plugins ( #3969 )
...
* Consolidate dbt-redshift
Start moving unit tests
fix typo; rm breakpoints
rm references to redshift
rm all references to redshift except `integration/100_rpc_test` 🙁
cleanup CI
fix tests!
fix docs test bc of schema version change
* Consolidate dbt-snowflake
Continue pulling apart Snowflake tests
Fixups
Rip out even more
* Consolidate dbt-bigquery
* Rm redshift, snowflake, bigquery from plugins/
* Comment out rs, sf, bq profiles in 100_rpc_test
* Add changelog, readme notes
Co-authored-by: Kyle Wigley <kyle@dbtlabs.com >
2021-10-02 10:30:09 -07:00
Emily Rockman
8ae232abe8
Merge pull request #3959 from dbt-labs/er/3899-models-key-mismatch
...
Er/3899 warn when node is specified under the wrong type & manifest schema disabled/_disabled consolidation
2021-10-01 16:36:47 -05:00
Emily Rockman
332d23c5eb
fixed issue with manifest schema changes not implemented for deepcopy overlooked arg order
2021-10-01 15:57:08 -05:00
Emily Rockman
5799973474
updated the disabled attr on the Manifest to be a dict instead of a list.
2021-10-01 15:57:08 -05:00
Emily Rockman
3d816d56ec
updated tests to reflect change of disabled from a list to a dict.
2021-10-01 15:57:08 -05:00
Emily Rockman
111f3c28f8
cleaned up exception changes
2021-10-01 15:57:08 -05:00
Emily Rockman
10aded793c
cleaned up exceptions
2021-10-01 15:57:08 -05:00
Emily Rockman
b5cc7b8dff
fixed warning logic and added test
2021-10-01 15:57:08 -05:00
Emily Rockman
449f042742
updated to warn when key doesn't match
2021-10-01 15:57:08 -05:00
Emily Rockman
66b70e025b
fixed forgotten breakpoint
2021-10-01 15:57:08 -05:00
Emily Rockman
578c6d6a20
fixed flake8 errors
2021-10-01 15:57:08 -05:00
Emily Rockman
64ce9d6aa4
removed _disabled to only use disabled
2021-10-01 15:57:08 -05:00
Emily Rockman
213ddedb85
tracking down diasabled model config
2021-10-01 15:57:08 -05:00
Emily Rockman
c96201c060
WIP
2021-10-01 15:57:08 -05:00
Gerda Shank
16b02f4f55
Merge pull request #3989 from dbt-labs/default_partial_parsing
...
[#3867 ] Turn on partial parsing by default
2021-10-01 16:24:44 -04:00
Gerda Shank
e0d2b02d46
[ #3867 ] Turn on partial parsing by default
2021-10-01 15:47:46 -04:00
Gerda Shank
65e76df6ec
Merge pull request #3982 from dbt-labs/3810_skip_pp_macros
...
[#3810 ] Skip partial parsing if certain macros have changed
2021-10-01 15:16:39 -04:00
Gerda Shank
052a3060d4
[ #3810 ] Skip partial parsing if certain macros have changed
2021-10-01 14:11:45 -04:00
Matthew McKnight
b65ae1ddde
Merge pull request #3974 from dbt-labs/McKnight-42/update-contributing
...
Mc knight 42/update contributing
2021-09-30 15:54:31 -05:00
Matthew McKnight
a8246ab1f1
Update CONTRIBUTING.md
...
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-09-30 15:47:44 -05:00
Nathaniel May
6854e67464
Merge pull request #3981 from dbt-labs/tracking-fix
...
(One Liner!) Add tracking that accidentally got removed
2021-09-30 10:38:21 -04:00
Jeremy Cohen
ca7c1fc4ad
Rename tests: singular + generic ( #3880 )
...
* Fix #3347
* Rename data->singular, schema->generic
* Rm schema, data flag + tag behavior
* Update test FQNs, compiled paths
* Add changelog entry
2021-09-30 06:56:58 -07:00
Nathaniel May
5dbc945f23
add tracking that accidentally got removed
2021-09-30 09:49:49 -04:00
Emily Rockman
655ff85dc9
Merge pull request #3937 from dbt-labs/3897-not-a-dict
...
Enact deprecation for not-a-dictionary to raise exception instead.
2021-09-29 21:20:45 -05:00
Emily Rockman
6a2ceaa073
Merge branch 'develop' into 3897-not-a-dict
2021-09-29 21:05:53 -05:00
Emily Rockman
e8fb29d185
Merge pull request #3909 from dbt-labs/3901-adapter-macro
...
Raise exception for adapter_macro instead of deprecation warning.
2021-09-29 21:05:09 -05:00
Emily Rockman
8443142f27
Merge branch 'develop' into 3901-adapter-macro
2021-09-29 20:46:44 -05:00
Emily Rockman
7ebe21dccb
Merge branch 'develop' into 3897-not-a-dict
2021-09-29 20:45:39 -05:00
kadero
c25b7a1143
Fix integration tests ( #3953 )
...
* Fix integration tests
* Update changelog.md
* Update changelog.md
* Bump up contributor note to 1.0.0
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com >
2021-09-29 13:57:11 -07:00
Nathaniel May
38eb46dfc3
Merge pull request #3939 from dbt-labs/static-parser-by-default
...
Turn static parser on by default with the ability to shut it off.
2021-09-29 16:01:52 -04:00
Jeremy Cohen
fe9ed9ccdd
Scrub strings, not AttributeErrors ( #3972 )
...
* Cast log msg to str before .replace()
* Skip assignment
2021-09-29 12:48:47 -07:00
Matthew McKnight
ff4e5219b1
cla rerun
2021-09-29 13:26:41 -05:00
Matthew McKnight
04632a008f
updated contribute.md to reflect swap over from CircleCI and Azure Pipelines to Github Actions for CI testing.
2021-09-29 13:26:18 -05:00
Nathaniel May
6925cebcf6
turn on static parser by default and add --no-static-parser flag
2021-09-28 13:21:11 -04:00
Jeremy Cohen
571beb13d9
Avoid infinite recursion in macro state check ( #3957 )
...
* Avoid infinite recursion in macro state check
* Add test case for infinite macro recursion
2021-09-27 08:43:29 -07:00
Emily Rockman
69cd82f483
added/cleaned tests, updated changelog
2021-09-27 09:26:20 -05:00
Emily Rockman
11e379280f
in progress
2021-09-27 09:24:46 -05:00
Emily Rockman
0018eb7db6
Raise exception for adapter_macro
2021-09-27 09:24:46 -05:00
Emily Rockman
154a682180
fixed meger bug
2021-09-27 08:58:21 -05:00
Emily Rockman
1b79a245e6
updated changlog
2021-09-27 08:38:08 -05:00
Emily Rockman
6b590122c7
updated exception
2021-09-27 08:37:25 -05:00
Emily Rockman
d5f632e6fd
enacted not-a-dict deprecation
2021-09-27 08:37:25 -05:00
Emily Rockman
2fc8e5e0b6
Merge pull request #3907 from dbt-labs/3900-execute-macro-release
...
removed the release parameter and associated deprecation
2021-09-27 07:16:37 -05:00
Emily Rockman
5ab07273ba
fixed flake error
2021-09-27 06:47:48 -05:00
Emily Rockman
19c9e5bfdf
Merge branch 'develop' into 3900-execute-macro-release
2021-09-27 06:42:11 -05:00
Emily Rockman
60794367a5
Merge pull request #3918 from dbt-labs/er/3898-column-quoting-unset
...
changes default quote_columns to True and removed associated deprecation
2021-09-27 06:39:26 -05:00
leahwicz
ea07729bbf
Bumping artifact versions for 0.21 ( #3945 )
...
* Bumping artifact versions for 0.21
* Adding sources bump
* Updating some tests
* Updated sources test
* Adding changelog update
2021-09-23 22:58:40 -04:00
Gerda Shank
c4370773f6
Merge pull request #3943 from dbt-labs/fix_flags_profiles_dir
...
[#3940 ] Use flags.PROFILES_DIR in a few more places
2021-09-23 13:51:10 -04:00
Gerda Shank
fda17b456e
[ #3940 ] Use flags.PROFILES_DIR in a few more places
2021-09-23 13:27:46 -04:00
Emily Rockman
bc3e1a0a71
fixed test and updated changelog
2021-09-23 12:01:53 -05:00
Emily Rockman
a06988706c
cleaned up snowflake test
2021-09-23 11:48:06 -05:00
Emily Rockman
ce73124bbf
fixed snowflake test
2021-09-23 11:48:06 -05:00
Emily Rockman
352c62f3c3
fixed missing import
2021-09-23 11:48:06 -05:00
Emily Rockman
81a51d3942
override for quote_seed_column methos in snowflake adapter
2021-09-23 11:48:06 -05:00
Emily Rockman
64fc3a39a7
changes default quote_columns to True and removed depracation
2021-09-23 11:48:05 -05:00
Emily Rockman
e5b6f4f293
updated changelog
2021-09-23 11:38:24 -05:00
Emily Rockman
d26e63ed9a
removed teh release parameter and associated deprications
2021-09-23 11:37:50 -05:00
dave-connors-3
f4f5d31959
Feature/catalog relational objects ( #3922 )
...
* filter to relational nodes
* cleanup
* flake formatting
* changelog
2021-09-23 08:54:05 -07:00
Jeremy Cohen
e7e12075b9
Fix batching for Snowflake seeds >10k rows ( #3942 )
...
* Call get_batch_size in snowflake__load_csv_rows
* Git ignore big csv. Update changelog
2021-09-23 08:49:52 -07:00
Emily Rockman
74dda5aa19
Merge pull request #3893 from dbt-labs/2798_enact_deprecations
...
removed deprecation for materialization-return and replaced with exception
2021-09-22 14:35:05 -05:00
Emily Rockman
092e96ce70
Merge branch 'develop' into 2798_enact_deprecations
2021-09-22 14:09:35 -05:00
Kyle Wigley
18102027ba
Pull in changes for the 0.21.0rc1 release ( #3935 )
...
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
2021-09-22 13:53:43 -05:00
Emily Rockman
f80825d63e
updated changelog
2021-09-22 12:55:49 -05:00
Kyle Wigley
9316e47b77
Pull in changes for the 0.21.0rc1 release ( #3935 )
...
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
2021-09-22 13:25:46 -04:00
Emily Rockman
f99cf1218a
fixed conflict
2021-09-22 11:36:22 -05:00
Emily Rockman
5871915ce9
Merge branch '2798_enact_deprecations' of https://github.com/dbt-labs/dbt into 2798_enact_deprecations
...
# Conflicts:
# test/integration/012_deprecation_tests/test_deprecations.py
2021-09-22 11:34:51 -05:00
Emily Rockman
5ce290043f
more explicit error check
2021-09-22 11:16:59 -05:00
Emily Rockman
080d27321b
removed deprication for materialization-return and replaced it with an exception
2021-09-22 11:16:59 -05:00
Gerda Shank
1d0936bd14
Merge pull request #3889 from dbt-labs/3886_pp_log_levels
...
[#3886 ] Tweak partial parsing log messages
2021-09-22 10:48:21 -04:00
Gerda Shank
706b8ca9df
Merge pull request #3839 from dbt-labs/2990_global_cli_flags
...
[#2990 ] Normalize global CLI args/flags
2021-09-22 10:47:54 -04:00
Nathaniel May
7dc491b7ba
Merge pull request #3936 from dbt-labs/regression-test-tweaks
...
Performance Regression Testing: Add timestamps to results and make filenames unique.
2021-09-22 10:18:24 -04:00
Gerda Shank
779c789a64
[ #2990 ] Normalize global CLI args/flags
2021-09-22 09:58:07 -04:00
Gerda Shank
409b4ba109
[ #3886 ] Tweak partial parsing log messages
2021-09-22 09:20:24 -04:00
Nathaniel May
59d131d3ac
add timestamps to results, and make filenames unique
2021-09-21 18:10:37 -04:00
Joel Labes
6563d09ba7
Add logging for skipped resources ( #3833 )
...
Add --greedy flag to subparser
Add greedy flag, override default behaviour when parsing union
Add greedy support to ls (I think?!)
That was suspiciously easy
Fix flake issues
Try adding tests with greedy support
Remove trailing whitespace
Fix return type for select_nodes
forgot to add greedy arg
Remove incorrectly expected test
Use named param for greedy
Pull alert_unused_nodes out into its own function
rename resources -> tests
Add expand_selection explanation of --greedy flag
Add greedy support to yaml selectors. Update warning, tests
Fix failing tests
Fix tests. Changelog
Fix flake8
Co-authored-by: Joel Labes c/o Jeremy Cohen <jeremy@dbtlabs.com >
2021-09-18 19:58:38 +02:00
Kyle Wigley
05dea18b62
update develop ( #3914 )
...
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
Co-authored-by: sungchun12 <sungwonchung3@gmail.com >
Co-authored-by: Drew Banin <drew@fishtownanalytics.com >
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com >
2021-09-17 14:07:42 -04:00
Nathaniel May
d7177c7d89
Merge pull request #3910 from dbt-labs/sample-more-exp-parser
...
sample exp parser more often
2021-09-17 13:34:14 -04:00
Gerda Shank
35f0fea804
Merge pull request #3888 from dbt-labs/skip_reloading_files
...
[#3563 ] Don't reload and validate schema files if they haven't changed
2021-09-17 13:32:01 -04:00
Gerda Shank
8953c7c533
[ #3563 ] Partial parsing: don't reload and validate schema files if they haven't changed
2021-09-17 13:27:25 -04:00
Jeremy Cohen
76c59a5545
Fix logging for default selector ( #3892 )
...
* Fix logging for default selector
* Fix flake8, mypy
2021-09-17 18:00:14 +02:00
Emily Rockman
237048c7ac
more explicit error check
2021-09-17 10:51:53 -05:00
Emily Rockman
30ff395b7b
removed deprication for materialization-return and replaced it with an exception
2021-09-17 10:51:53 -05:00
Nathaniel May
5c0a31b829
sample exp parser more often
2021-09-17 11:42:35 -04:00
Nathaniel May
243bc3d41d
Merge pull request #3877 from dbt-labs/exp-parser-detect-macros
...
Make ModelParser detect macro overrides for ref, source, and config.
2021-09-17 11:39:16 -04:00
Sam Debruyn
67b594a950
group by column_name in accepted_values ( #3906 )
...
* group by column_name in accepted_values
Group by index is not ANSI SQL and not supported in every database engine (e.g. MS SQL). Use group by column_name in shared code.
* update changelog
2021-09-17 17:08:41 +02:00
Kyle Wigley
2493c21649
Cleanup CHANGELOG ( #3902 )
2021-09-17 09:22:37 -04:00
Kyle Wigley
d3826e670f
Build task arg parity ( #3884 )
2021-09-16 16:21:37 -04:00
Drew Banin
4b5b1696b7
Merge pull request #3894 from dbt-labs/feature/source-freshness-timing-in-artifacts
...
Add timing and thread info to sources.json artifact
2021-09-16 14:59:18 -04:00
Drew Banin
abb59ef14f
add changelog entry
2021-09-16 13:54:04 -04:00
Drew Banin
3b7c2816b9
( #3804 ) Add timing and thread info to sources.json artifact
2021-09-16 13:28:08 -04:00
Teddy
484517416f
3448 user defined default selection criteria ( #3875 )
...
* Added selector default
* Updated code from review feedback
* Fix schema.yml location
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-09-16 16:13:59 +02:00
Nathaniel May
39447055d3
add macro override detection for ref, source, and config for ModelParser
2021-09-15 12:09:47 -04:00
Kyle Wigley
95cca277c9
Handle failing tests for build command ( #3792 )
...
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-09-15 10:39:19 -04:00
Christophe Oudar
96083dcaf5
add support for execution project for BigQuery adapter ( #3707 )
...
* add support for execution project for BigQuery adapter
* Add integration test for BigQuery execution_project
* review changes
* Update changelog
2021-09-15 14:09:08 +02:00
Emily Rockman
75b4cf691b
Merge pull request #3879 from dbt-labs/emmyoop/update-contributing
...
Fixed typo and added some more setup details to CONTRIBUTING.md
2021-09-14 09:35:12 -05:00
Emily Rockman
7c9171b00b
Fixed typo and added some more setup details
2021-09-13 14:03:02 -05:00
Kyle Wigley
3effade266
deepcopy args when passed down to rpc pask ( #3850 )
2021-09-10 16:34:57 -04:00
Jeremy Cohen
44e7390526
Specify macro_namespace of global_project dispatch macros ( #3851 )
...
* Specify macro_namespace of global_project dispatch macros
* Dispatch get_custom_alias, too
* Add integration tests
* Add changelog entry
2021-09-09 12:59:39 +02:00
Jeremy Cohen
c141798abc
Use a macro for where subquery in tests ( #3859 )
...
* Use a macro for where subquery in tests
* Fix existing tests
* Add test case reproducing #3857
* Add changelog entry
2021-09-09 12:38:09 +02:00
sungchun12
df7ec3fb37
Fix dbt deps sorting behavior for non-standard version semantics ( #3856 )
...
* robust sorting and default to original version str
* more unique version semantics
* add another non-standard version example
* fix mypy issue
2021-09-09 12:13:09 +02:00
AndreasTA-AW
90e5507d03
#3682 Changed how tables and views are generated to be able to use differen… ( #3691 )
...
* Changed how tables and views are generated to be able to use different options
* 3682 added unit tests
* 3682 had conflict in changelog and became a bit messy
* 3682 Tested to add default kms to dataset and accidently pushed the changes
2021-09-09 12:01:02 +02:00
leahwicz
332d3494b3
Adding ADR directory, guidelines, first one ( #3844 )
...
* Adding ADR README
* Adding perf testing ADR
* fill in adr sections
Co-authored-by: Nathaniel May <nathaniel.may@fishtownanalytics.com >
2021-09-07 14:05:21 -04:00
Anna Filippova
6393f5a5d7
Feature: Add support for Package name changes on the Hub ( #3825 )
...
* Add warning about new package name
* Update CHANGELOG.md
* make linter happy
* Add warning about new package name
* Update CHANGELOG.md
* make linter happy
* move warnings to deprecations
* Update core/dbt/clients/registry.py
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
* add comments for posterity
* Update core/dbt/deprecations.py
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
* add deprecation test
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-09-07 11:53:02 -04:00
Kyle Wigley
ce97a9ca7a
explicitly define __init__ for Profile class ( #3855 )
2021-09-07 10:38:51 -04:00
Jeremy Cohen
9af071bfe4
Add adapter_unique_id to invocation tracking ( #3796 )
...
* Add properties + methods for adapter_unique_id
* Turn on tracking
2021-09-03 16:38:20 +02:00
sungchun12
45a41202f3
fix prerelease imports with loose version semantics ( #3852 )
...
* fix prereleases imports with looseversion
* update for non-standard versions
2021-09-02 17:52:36 +02:00
Kyle Wigley
9768999ca1
Only set single_threaded for the rpc list task, not the cli list task ( #3848 )
2021-09-02 10:11:39 -04:00
juma-adoreme
fc0d11c0a5
Parametrize key selection for the list task ( #3838 )
...
* Parametrize key selectinf for list task
* Remove trailing whitespace
* Add output_keys to RPC List Parameters
* Move up changelog entry, add contributor note
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-09-02 14:58:00 +02:00
Joel Labes
e6344205bb
Add colourful count of pass/fail tests in dbt debug ( #3832 )
...
* Add colourful count of pass/fail tests in dbt debug
* Remove number of checks, move error messages into shared list
* Fix flake issues
* Update CHANGELOG.md
2021-09-02 12:15:03 +02:00
Jason Gluck
9d7a6556ef
configurable postgres connect timeout ( #3582 )
...
* configurable postgres connect timeout
* changelog for #3582
* test default and change connect_timeout
* Move up contributor note in changelog
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-08-31 19:45:31 +02:00
Daniel Bartley
15f4add0b8
Add target_project and target_dataset config aliases for snapshots on BigQuery ( #3834 )
...
* add bq alias for target_project and target_dataset
* Update CHANGELOG.md
add #3694 to changelog
* Update CHANGELOG.md
Be more specific about the change to bigquery synonym for schema only.
* Set integration test bigquery configs to use alias
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-08-31 16:08:20 +02:00
Anders
464becacd0
fewer adapters will need to re-implemnt basic_load_csv_rows ( #3623 )
...
* fewer adapters will need to re-implemnt basic_load_csv_rows
* hack version
* reordering per convention
* make redundant basic_load_csv_rows
* for next version
* Update core/dbt/include/global_project/macros/materializations/seed/seed.sql
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
* Move up changelog entry
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-08-31 15:47:36 +02:00
sungchun12
51a76d0d63
Better dbt packages version logging to aid in upgrading outdated packages ( #3759 )
...
* start blueprinting changes
* extend registry handler for latest package version
* conditional logging for latest version
* remove todo
* add conditional logging
* Upgrades is clearer
* update if elif conditions and log msg
* remove TODO
* fix flake8 errors
* blueprint unit tests
* conditions specific to hub registry
* 1 passing test for get latest version
* DRY method calls
* move version latest to hub only
* add a new line
* remove other draft tests
* update changelog
* update log language for clarity
* pass flake8
* fix changelog
* Update test/unit/test_deps.py
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
* update changelog
* remove hub language
* sort for latest version and include prereleases
* fix flake8
* resolves another issue
* fix prerelease string formatting
* fix broken test
* update logging to past tense
* built-in version sorting
* handle prereleases for latest version checks
* get version latest unit test based on prerelease
* update unit test for sorting functionality
* consistent test names
* fix flake8
* clean up contributors list
* simplify if else logic
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-08-31 10:31:45 +02:00
Slava Kalashnikov
052e54d43a
BigQuery copy materialization enhancement ( #3606 )
...
* Change BigQuery copy materialization
Change BigQuery copy materialization macros to copy data from several sources into single target
* Change BigQuery copy materialization
Change BigQuery connections.py to copy data from several sources into single target via copy materialization
* Change BigQuery copy materialization
Test to check default value of `copy_materialization` if it is absent in config
* Change BigQuery copy materialization
Update changelog
* Update changelog
* Var renaming + test addition
* Changelog updated
* Changelog updated
* Fix test for copy table
* Update test_bigquery_adapter.py
* Update test_bigquery_adapter.py
* Update impl.py
* Update connections.py
* Update test_bigquery_adapter.py
* Update test_bigquery_adapter.py
* Update connections.py
* Align calls from mock and from adapter
* Split long code ilnes
* Create additional.sql
* Update copy_as_several_tables.sql
* Update schema.yml
* Update copy.sql
* Update connections.py
* Update test_bigquery_copy_models.py
* Add contributor
2021-08-30 16:28:35 +02:00
Kyle Wigley
9e796671dd
Update workflow concurrency ( #3824 )
2021-08-26 17:13:54 -04:00
Kyle Wigley
a9a6254f52
Address unexpected cancelled CI workflows and stop blocking Postgres integration tests ( #3813 )
2021-08-26 10:37:50 -04:00
Kyle Wigley
8b3a09c7ae
Run postgres integration tests when dev dependency changes ( #3819 )
2021-08-26 10:36:24 -04:00
Jeremy Cohen
6aa4d812d4
Rewrite generic tests to support column expressions ( #3812 )
...
* Rewrite generic tests to support column expressions, too
* Fix naming
2021-08-26 10:30:03 -04:00
Kyle Wigley
07fa719fb0
Revert "Bump freezegun from 0.3.12 to 1.1.0" ( #3818 )
...
This reverts commit 650b34ae24 .
2021-08-26 10:11:56 -04:00
dependabot[bot]
650b34ae24
Bump freezegun from 0.3.12 to 1.1.0 ( #3206 )
...
Bumps [freezegun](https://github.com/spulec/freezegun ) from 0.3.12 to 1.1.0.
- [Release notes](https://github.com/spulec/freezegun/releases )
- [Changelog](https://github.com/spulec/freezegun/blob/master/CHANGELOG )
- [Commits](https://github.com/spulec/freezegun/compare/0.3.12...1.1.0 )
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-08-25 18:18:37 -04:00
dependabot[bot]
0a935855f3
Update sqlparse requirement from <0.4,>=0.2.3 to >=0.2.3,<0.5 in /core ( #3074 )
...
Updates the requirements on [sqlparse](https://github.com/andialbrecht/sqlparse ) to permit the latest version.
- [Release notes](https://github.com/andialbrecht/sqlparse/releases )
- [Changelog](https://github.com/andialbrecht/sqlparse/blob/master/CHANGELOG )
- [Commits](https://github.com/andialbrecht/sqlparse/compare/0.2.3...0.4.1 )
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-08-25 09:59:36 -04:00
dependabot[bot]
d500aae4dc
Bump ubuntu from 18.04 to 20.04 ( #3073 )
...
Bumps ubuntu from 18.04 to 20.04.
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-08-25 09:35:24 -04:00
Kyle Wigley
370d3e746d
Remove fishtown-analytics references 😢 ( #3801 )
2021-08-25 09:24:41 -04:00
Kyle Wigley
ab06149c81
Moving CI to GitHub actions ( #3669 )
...
* test
* test test
* try this again
* test actions in same repo
* nvm revert
* formatting
* fix sh script for building dists
* fix windows build
* add concurrency
* fix random 'Cannot track experimental parser info when active user is None' error
* fix build workflow
* test slim ci
* has changes
* set up postgres for other OS
* update descriptions
* turn off python3.9 unit tests
* add changelog
* clean up todo
* Update .github/workflows/main.yml
* create actions for common code
* temp commit to test
* cosmetic updates
* dev review feedback
* updates
* fix build checks
* rm auto formatting changes
* review feeback: update order of script for setting up postgres on macos runner
* review feedback: add reasoning for not using secrets in workflow
* review feedback: rm unnecessary changes
* more review feedback
* test pull_request_target action
* fix path to cli tool
* split up lint and unit workflows for clear resposibilites
* rm `branches-ignore` filter from pull request trigger
* testing push event
* test
* try this again
* test actions in same repo
* nvm revert
* formatting
* fix windows build
* add concurrency
* fix build workflow
* test slim ci
* has changes
* set up postgres for other OS
* update descriptions
* turn off python3.9 unit tests
* add changelog
* clean up todo
* Update .github/workflows/main.yml
* create actions for common code
* cosmetic updates
* dev review feedback
* updates
* fix build checks
* rm auto formatting changes
* review feedback: add reasoning for not using secrets in workflow
* review feedback: rm unnecessary changes
* more review feedback
* test pull_request_target action
* fix path to cli tool
* split up lint and unit workflows for clear resposibilites
* rm `branches-ignore` filter from pull request trigger
* test dynamic matrix generation
* update label logic
* finishing touches
* align naming
* pass opts to pytest
* slim down push matrix, there are a lot of jobs
* test bump num of proc
* update matrix for all event triggers
* handle case when no changes require integration tests
* dev review feedback
* clean up and add branch name for testing
* Add test results publishing as artifact (#3794 )
* Test failures file
* Add testing branch
* Adding upload steps
* Adding date to name
* Adding to integration
* Always upload artifacts
* Adding adapter type
* Always publish unit test results
* Adding comments
* rm unecessary env var
* fix changelog
* update job name
* clean up python deps
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
2021-08-24 17:12:42 -04:00
Gerda Shank
e72895c7c9
Merge pull request #3791 from dbt-labs/3210_select_equals_model
...
[#3210 ] Make --models and --select synonyms, except for 'ls'
2021-08-24 14:44:24 -04:00
Gerda Shank
fe4a67daa4
Use 'select' instead of 'models' for internal args processing and RPC
2021-08-24 13:55:37 -04:00
leahwicz
09ea989d81
Retry GitHub download failures ( #3729 )
...
* Retry GitHub download failures
* Refactor and add tests
* Fixed linting and added comment
* Fixing unit test assertRaises
Co-authored-by: Kyle Wigley <kyle@fishtownanalytics.com >
* Fixing casing
Co-authored-by: Kyle Wigley <kyle@fishtownanalytics.com >
* Changing to use partial for function calls
Co-authored-by: Kyle Wigley <kyle@fishtownanalytics.com >
2021-08-24 13:35:09 -04:00
leahwicz
7fa14b6948
Fixing changelog ( #3776 )
2021-08-23 13:19:58 -04:00
Gerda Shank
d4974cd35c
[ #3210 ] Make --models and --select synonyms, except for 'ls'
2021-08-23 11:40:59 -04:00
Kyle Wigley
459178811b
rm git backports for previous debian release, use git package ( #3785 )
2021-08-22 21:20:34 -04:00
Snyk bot
b37f6a010e
fix: docker/Dockerfile to reduce vulnerabilities ( #3771 )
...
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-DEBIAN10-SQLITE3-537598
- https://snyk.io/vuln/SNYK-DEBIAN10-SYSTEMD-345386
- https://snyk.io/vuln/SNYK-DEBIAN10-SYSTEMD-345386
- https://snyk.io/vuln/SNYK-DEBIAN10-SYSTEMD-345391
- https://snyk.io/vuln/SNYK-DEBIAN10-SYSTEMD-345391
2021-08-19 09:26:53 -04:00
Gerda Shank
e817164d31
Merge pull request #3767 from dbt-labs/3764_analysis_descriptions
...
[#3764 ] Fix bug in analysis patch application
2021-08-18 09:05:10 -04:00
Gerda Shank
09ce43edbf
[ #3764 ] Fix bug in analysis patch application
2021-08-17 18:07:35 -04:00
sungchun12
2980cd17df
Fix/bigquery job label length ( #3703 )
...
* add blueprints to resolve issue
* revert to previous version
* intentionally failing test
* add imports
* add validation in existing function
* add passing test for length validation
* add current sanitized label
* remove duplicate var
* Make logging output 2 lines
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
* Raise RuntimeException to better handle error
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
* update test
* fix flake8 errors
* update changelog
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-08-17 14:54:31 -04:00
Gerda Shank
8c804de643
Merge pull request #3758 from dbt-labs/3757_pp_version_mismatch
...
[#3757 ] Produce better information about partial parsing version mismatches
2021-08-17 13:23:23 -04:00
Gerda Shank
c8241b87e6
[ #3757 ] Produce better information about partial parsing version
...
mismatches.
2021-08-17 12:48:20 -04:00
Gerda Shank
f204d24ed8
Merge pull request #3616 from dbt-labs/config_in_schema_files
...
Configs in schema files
2021-08-17 12:18:04 -04:00
Gerda Shank
d5461ccd8b
[ #2401 ] Configs in schema files
2021-08-17 11:50:08 -04:00
Gerda Shank
a20d2d93d3
Merge pull request #3750 from dbt-labs/fix_remove_tests
...
[#3711 ] Check that test unique_id exists in nodes when removing
2021-08-16 09:06:36 -04:00
Gerda Shank
57e1eec165
[ #3711 ] Check that test unique_id exists in nodes when removing
2021-08-13 17:23:36 -04:00
Nathaniel May
d2dbe6afe4
Merge pull request #3739 from dbt-labs/perf-testing-tweak
...
Bump minimum performance runs to 20
2021-08-13 14:05:10 -04:00
Gerda Shank
72eb163223
Merge pull request #3733 from dbt-labs/pp_trap_errors
...
Trap partial parsing errors and switch to full reparse on exceptions
2021-08-13 13:54:40 -04:00
Gerda Shank
af16c74c3a
[ #3725 ] Switch to full reparse on partial parsing exceptions. Log and
...
report exception information.
2021-08-13 13:38:47 -04:00
Kyle Wigley
664f6584b9
add missing versions and format ( #3738 )
2021-08-13 13:31:38 -04:00
Nathaniel May
76fd3bdf8c
minimum performance runs to 20
2021-08-13 13:20:30 -04:00
Jeremy Cohen
b633adb881
Use is_relational check for schema caching ( #3716 )
...
* Use is_relational check for schema caching
* Fix flake8
* Update changelog
2021-08-12 18:18:28 -04:00
Jeremy Cohen
b6e534cdd0
Feature: state:modified.macros ( #3559 )
...
* First cut at state:modified for macro changes
* First cut at state:modified subselectors
* Update test_graph_selector_methods
* Fix flake8
* Fix mypy
* Update 062_defer_state_test/test_modified_state
* PR feedback. Update changelog
2021-08-12 17:21:48 -04:00
Nathaniel May
1dc4adb86f
Merge pull request #3732 from dbt-labs/perf-test-project-swap
...
Swap dummy perf testing projects for a real one
2021-08-12 10:27:20 -04:00
Nathaniel May
0a4d7c4831
Merge pull request #3731 from dbt-labs/perf-ci-quickfix
...
remove pr trigger for perf workflow
2021-08-12 10:25:13 -04:00
Nathaniel May
ad67e55d74
swapping dummy perf testing projects for real one
2021-08-11 14:24:26 -04:00
Nathaniel May
2fae64a488
remove pr trigger for perf workflow
2021-08-11 14:14:08 -04:00
Nathaniel May
1a984601ee
Merge pull request #3602 from dbt-labs/performance-regression-testing
...
Add Performance Regression Testing [Rust]
2021-08-11 10:44:51 -04:00
Jeremy Cohen
454168204c
Add build RPC method ( #3674 )
...
* Add build RPC method
* Add rpc test, some required flags
* Fix flake8
* PR feedback
* Update changelog [skip ci]
* Do not skip CI when rebasing
2021-08-10 10:51:43 -04:00
Drew Banin
43642956a2
Serialize Undefined values to JSON for rpc requests ( #3687 )
...
* (#3464 ) Serialize Undefined values to JSON for rpc requests
* Update changelog, fix typo
2021-08-09 21:26:09 -04:00
Nathaniel May
1fe53750fa
add more future work
2021-08-09 12:55:42 -04:00
Nathaniel May
8609c02383
minor change
2021-08-09 12:53:53 -04:00
Nathaniel May
355b0c496e
remove unnecessary newline printing
2021-08-09 12:53:07 -04:00
Nathaniel May
cd6894acf4
add to future work
2021-08-09 12:52:53 -04:00
Nathaniel May
b90b3a9c19
avoid abandoned destructors by refactoring usage of exit
2021-08-06 14:46:10 -04:00
leahwicz
e7b8488be8
Remove converter.py since not used anymore ( #3699 )
2021-08-05 15:27:56 -04:00
Nathaniel May
06cc0c57e8
changelog
2021-08-05 12:27:21 -04:00
Nathaniel May
87072707ed
point to manifest
2021-08-05 11:40:48 -04:00
Nathaniel May
ef63319733
add comment
2021-08-05 11:39:19 -04:00
Nathaniel May
2068dd5510
fmt
2021-08-05 11:31:19 -04:00
Nathaniel May
3e1e171c66
add test for regression calculation
2021-08-05 11:31:00 -04:00
Nathaniel May
5f9ed1a83c
run twice a day
2021-08-05 11:15:40 -04:00
Nathaniel May
3d9e54d970
add fmt check
2021-08-05 11:14:30 -04:00
Nathaniel May
52a0fdef6c
fmt
2021-08-05 11:10:07 -04:00
Nathaniel May
d9b02fb0a0
add lots of comments
2021-08-05 11:03:13 -04:00
Nathaniel May
6c8de62b24
up stddev threshold
2021-08-05 09:48:33 -04:00
Nathaniel May
2d3d1b030a
rename to dummy project
2021-08-05 09:47:34 -04:00
Nathaniel May
88acf0727b
remove clone
2021-08-04 17:50:32 -04:00
Nathaniel May
02839ec779
iter instead of into_iter
2021-08-04 17:48:13 -04:00
Nathaniel May
44a8f6a3bf
more reference improvements
2021-08-04 17:46:45 -04:00
Nathaniel May
751ea92576
make the happy path use references and clone in the exception paths
2021-08-04 17:33:29 -04:00
Nathaniel May
02007b3619
more reference handling
2021-08-04 17:05:23 -04:00
Nathaniel May
fe0b9e7ef5
refactor to use references better
2021-08-04 17:00:16 -04:00
Nathaniel May
4b1c6b51f9
fix spacing
2021-08-04 16:39:53 -04:00
Nathaniel May
0b4689f311
address PR feedback
2021-08-04 15:14:37 -04:00
Nathaniel May
b77eff8f6f
errors to warnings in ci
2021-08-04 13:34:46 -04:00
Nathaniel May
2782a33ecf
minor refactor
2021-08-04 13:31:48 -04:00
Nathaniel May
94c6cf1b3c
remove some clones
2021-08-04 13:19:39 -04:00
Nathaniel May
3c8daacd3e
refactor for simpler flow
2021-08-04 13:12:01 -04:00
Nathaniel May
2f9907b072
remove one more unwrap
2021-08-04 12:34:57 -04:00
Nathaniel May
287c4d2b03
revamp exception hierarchy
2021-08-04 12:15:29 -04:00
Nathaniel May
ba9d76b3f9
make final json human readable
2021-08-04 10:11:09 -04:00
Jeremy Cohen
0efaaf7daf
Fix typo [skip ci]
2021-08-04 09:50:11 -04:00
Drew Banin
9ae7d68260
Merge pull request #3686 from dbt-labs/fix/cleanup-audit-integration-tests
...
Fix: Drop audit schema tests in tearDown for test suite
2021-08-03 19:54:36 -04:00
Nathaniel May
486afa9fcd
upload results even when regressions are detected
2021-08-03 18:04:22 -04:00
Nathaniel May
1f189f5225
added small paragraph to readme
2021-08-03 17:59:53 -04:00
Nathaniel May
580b1fdd68
minor print change
2021-08-03 17:56:30 -04:00
Nathaniel May
bad0198a36
minor readme changes
2021-08-03 17:55:03 -04:00
Nathaniel May
252280b56e
write out results as artifact
2021-08-03 17:34:10 -04:00
Nathaniel May
64bf9c8885
test fix
2021-08-03 17:04:13 -04:00
Nathaniel May
935c138736
type gymnastics
2021-08-03 17:03:37 -04:00
Nathaniel May
5891b59790
better variable names
2021-08-03 16:52:28 -04:00
Nathaniel May
4e020c3878
enforce branch names at calculation time
2021-08-03 16:50:08 -04:00
Nathaniel May
3004969a93
move measure io to main
2021-08-03 16:26:29 -04:00
Nathaniel May
873e9714f8
move io operations to main
2021-08-03 16:22:02 -04:00
Nathaniel May
fe24dd43d4
return all calculations not just regressions
2021-08-03 16:10:16 -04:00
Nathaniel May
ed91ded2c1
branches named dev and baseline
2021-08-03 15:04:05 -04:00
Nathaniel May
757614d57f
add stddev regression
2021-08-03 14:59:21 -04:00
Nathaniel May
faff8c00b3
group by run only
2021-08-03 14:37:49 -04:00
Github Build Bot
45fe76eef4
Merge remote-tracking branch 'origin/releases/0.21.0b1' into develop
2021-08-03 18:09:56 +00:00
Nathaniel May
80244a09fe
add error for groups that are not two elements large
2021-08-03 14:00:30 -04:00
Github Build Bot
ea772ae419
Release dbt v0.21.0b1
2021-08-03 17:30:32 +00:00
Drew Banin
c68fca7937
Fix: Drop audit schema tests in tearDown for test suite
2021-08-03 13:24:54 -04:00
Nathaniel May
37e86257f5
point to the downloaded artifacts correctly
2021-08-03 12:57:19 -04:00
Nathaniel May
c182c05c2f
error when there are no results to process
2021-08-03 12:48:21 -04:00
Nathaniel May
b02875a12b
add debug line
2021-08-03 12:35:32 -04:00
Nathaniel May
03332b2955
more gracefully exit than unwrapping
2021-08-03 12:32:56 -04:00
Nathaniel May
f1f99a2371
add name to action
2021-08-03 12:15:44 -04:00
Nathaniel May
95116dbb5b
fix measure call
2021-08-03 12:12:25 -04:00
Nathaniel May
868fd64adf
download correct artifact
2021-08-03 12:08:00 -04:00
Nathaniel May
2f7ab2d038
unify rust apps as subcommands
2021-08-03 12:04:45 -04:00
Jeremy Cohen
159e79ee6b
Update changelog in advance of v0.21.0b1 ( #3678 )
...
* Fixup Changelog
* More updates [skip ci]
2021-08-02 20:08:22 -04:00
Nathaniel May
3d4a82cca2
add comments
2021-08-02 18:09:31 -04:00
Nathaniel May
6ba837d73d
fix download artifact name
2021-08-02 18:03:04 -04:00
Nathaniel May
f4775d7673
change job name
2021-08-02 17:56:29 -04:00
Nathaniel May
429396aa02
move step dependencies around
2021-08-02 17:55:26 -04:00
Nathaniel May
8a5e9b71a5
add some output to comparitor
2021-08-02 17:50:33 -04:00
Nathaniel May
fa78102eaf
add comparison to workflow
2021-08-02 17:23:30 -04:00
Nathaniel May
5466d474c5
add test for exception display
2021-08-02 17:07:41 -04:00
Nathaniel May
80951ae973
fmt
2021-08-02 16:34:47 -04:00
Nathaniel May
d5662ef34c
wrap up comparsion logic
2021-08-02 16:33:43 -04:00
leahwicz
57783bb5f6
Adding issue templates for different release types ( #3644 )
...
Co-authored-by: Kyle Wigley <kyle@fishtownanalytics.com >
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-08-02 12:50:49 -04:00
Nathaniel May
d73ee588e5
Merge pull request #3637 from dbt-labs/experimental-parser-fix
...
make experimental parser respect config merge behavior
2021-08-02 10:03:42 -04:00
Nathaniel May
40089d710b
experimental parser respects config merge behavior
2021-08-02 09:38:30 -04:00
Jeremy Cohen
6ec61950eb
Handle exception from tracker.flush() ( #3661 )
2021-08-02 08:25:41 -04:00
Gerda Shank
72c831a80a
Merge pull request #3659 from dbt-labs/pp_internal_macro_processing
...
[#3636 ] Check for unique_ids when recursively removing macros
2021-07-30 15:34:14 -04:00
Gerda Shank
929931a26a
Merge pull request #3654 from dbt-labs/change_config_call_handling
...
Switch from config_call list to config_call_dict dictionary
2021-07-30 14:08:30 -04:00
Gerda Shank
577e2438c1
[ #3636 ] Check for unique_ids when recursively removing macros
2021-07-30 14:01:40 -04:00
Kyle Wigley
2679792199
Add tracking event for full re-parse reasoning ( #3652 )
...
* add tracking event for full reparse reason
* update changelog
2021-07-30 09:39:09 -04:00
Kyle Wigley
2adf982991
update links to dbt repo ( #3521 )
2021-07-30 08:46:58 -04:00
Gerda Shank
1fb4a7f428
Switch from config_call list to config_call_dict dictionary
2021-07-29 18:46:59 -04:00
Kyle Wigley
30e72bc5e2
Use SchemaParser render context to render test configs ( #3646 )
...
* use available context when rendering test configs
* add test
* update changelog
2021-07-29 12:59:48 -04:00
Jeremy Cohen
35645a7233
Include dbt-docs changes for 0.20.1-rc1 ( #3643 )
2021-07-29 09:56:04 -04:00
Gerda Shank
d583c8d737
Merge pull request #3632 from dbt-labs/pp_delete_schema_macro_patch
...
[#3627 ] Improve findability of macro_patches, schedule right macro file for processing
2021-07-28 17:49:27 -04:00
Gerda Shank
a83f00c594
[ #3627 ] Improve findability of macro_patches, schedule right macro file
...
for processing
2021-07-28 17:27:42 -04:00
Nathaniel May
45bb955b55
still wip but compiles
2021-07-28 15:54:04 -04:00
Daniele Frigo
c448702c1b
Use old_relation for renaming in default materializations ( #3547 )
...
* table and view materializations should rename from old_relation to manage changes from view to table and reverse
* edited changelog
* edited changelog
* Update CHANGELOG.md
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2021-07-28 06:59:27 -04:00
Niall Woodward
558a6a03ac
Fix PR link in changelog ( #3639 )
...
Fix a typo introduced in https://github.com/dbt-labs/dbt/pull/3624
2021-07-28 06:51:45 -04:00
Niall Woodward
52ec7907d3
dbt deps prerelease install bugs + add install-prerelease parameter to packages.yml ( #3624 )
...
* Fix dbt deps prerelease install bugs
* Add install-prerelease parameter to hub packages in packages.yml
2021-07-27 21:59:46 -04:00
Jeremy Cohen
792f39a888
Snowflake: no transactions, except for DML ( #3510 )
...
* Rm Snowflake txnal logic. Explicit for DML
* Be less clever. Update create_or_replace_view()
* Seed DML as well
* Changelog entry
* Fix unit test
* One semicolon can change the world
2021-07-27 18:13:35 -04:00
Gerda Shank
16264f58c1
Merge pull request #3621 from dbt-labs/pp_macro_link_processing_error
...
[#3584 ] Partial parsing: handle source tests when changing test macro
2021-07-27 16:59:26 -04:00
Nathaniel May
2317c0c3c8
Merge pull request #3630 from dbt-labs/nate-3568
...
fix awkward exception being raised by a yml file with all comments
2021-07-27 16:50:56 -04:00
Gerda Shank
3c09ab9736
[ #3584 ] Partial parsing: handle source tests when changing test macro
2021-07-27 16:34:23 -04:00
Gerda Shank
f10dc0e1b3
Merge pull request #3618 from dbt-labs/pp_yaml_version
...
[#3567 ] Fix partial parsing error with version key if previous file is empty
2021-07-27 16:30:06 -04:00
leahwicz
634bc41d8a
Secret scrubbing for env variables ( #3617 )
...
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-07-27 16:06:10 -04:00
Gerda Shank
d7ea3648c6
[ #3567 ] Fix partial parsing error with version key if previous file is
...
empty
2021-07-27 15:38:52 -04:00
Gerda Shank
e5c8e19ff2
Merge pull request #3619 from dbt-labs/model_config_iterator
...
[#3573 ] Put back config iterator for backwards compatibility
2021-07-27 15:34:51 -04:00
Kyle Wigley
4ddba7e44c
--wip--
2021-07-27 12:48:33 -04:00
Kyle Wigley
37b31d10c8
add derived traits to structs
2021-07-27 11:28:32 -04:00
Nathaniel May
93cf1f085f
handle None return value from yaml loading
2021-07-27 10:59:27 -04:00
Gerda Shank
a84f824a44
[ #3573 ] Put back config iterator for backwards compatibility
2021-07-26 17:56:35 -04:00
Kyle Wigley
9c58f3465b
Fix flaky test related to tracking events ( #3604 )
...
* skip all tracking event testing
* Turn off tracking in tests that hits model parsing code path
fix other random test that fails because global tracking.current_user exists but is null
* pytest did not respect skip mark
* fix gh actions
2021-07-26 16:55:16 -04:00
Gerda Shank
0e3778132b
Merge pull request #3620 from dbt-labs/pp_already_removed_node
...
If SQL file already scheduled for parsing, don't reprocess
2021-07-26 15:49:10 -04:00
Jeremy Cohen
72722635f2
Fix error handling in dbt build ( #3608 )
...
* RunTask -> BuildTask
* Add test, changelog entry
2021-07-25 22:15:13 -04:00
Gerda Shank
a4c7c7fc55
If SQL file already scheduled for parsing, don't reprocess
2021-07-24 15:43:54 -04:00
Nathaniel May
2bad73eead
Merge pull request #3610 from dbt-labs/derp-fix
...
fixing typo in test
2021-07-23 13:14:55 -04:00
Kyle Wigley
c8bc25d11a
add struct
2021-07-22 17:05:08 -04:00
Kyle Wigley
4c06689ff5
create a tuple of measurements to group results by
2021-07-22 16:54:57 -04:00
Kyle Wigley
a45c9d0192
add new arg for runner for branch name
2021-07-22 16:24:46 -04:00
Kyle Wigley
34e2c4f90b
fix results output dir
2021-07-22 16:16:21 -04:00
Kyle Wigley
c0e2023c81
update profiles dir path
2021-07-22 15:52:28 -04:00
Nathaniel May
108b55bdc3
add regression type
2021-07-22 13:50:17 -04:00
Nathaniel May
a29367b7fe
read files and parse json
2021-07-22 12:30:35 -04:00
Nathaniel May
1d7e8349ed
draft of comparison
2021-07-22 12:01:30 -04:00
Nathaniel May
67c194dcd1
fixing typo in test
2021-07-22 09:53:26 -04:00
Nathaniel May
75d3d87d64
fix warmup syntax
2021-07-21 18:00:07 -04:00
Nathaniel May
4ff3f6d4e8
moved config dir out of projects dir
2021-07-21 17:35:54 -04:00
Nathaniel May
d0773f3346
warmup fs caches
2021-07-21 17:34:22 -04:00
Nathaniel May
ee58d27d94
use a profiles.yml file
2021-07-21 17:28:28 -04:00
Nathaniel May
9e3da391a7
draft of regression testing workflow
2021-07-21 17:01:01 -04:00
matt-winkler
bd7010678a
Feature: on_schema_change for incremental models ( #3387 )
...
* detect and act on schema changes
* update incremental helpers code
* update changelog
* fix error in diff_columns from testing
* abstract code a bit further
* address matching names vs. data types
* Update CHANGELOG.md
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
* updates from Jeremy's feedback
* multi-column add / remove with full_refresh
* simple changes from JC's feedback
* updated for snowflake
* reorganize postgres code
* reorganize approach
* updated full refresh trigger logic
* fixed unintentional wipe behavior
* catch final else condition
* remove WHERE string replace
* touch ups
* port core to snowflake
* added bigquery code
* updated impacted unit tests
* updates from linting tests
* updates from linting again
* snowflake updates from further testing
* fix logging
* clean up incremental logic
* updated for bigquery
* update postgres with new strategy
* update nodeconfig
* starting integration tests
* integration test for ignore case
* add test for append_new_columns
* add integration test for sync
* remove extra tests
* add unique key and snowflake test
* move incremental integration test dir
* update integration tests
* update integration tests
* Suggestions for #3387 (#3558 )
* PR feedback: rationalize macros + logging, fix + expand tests
* Rm alter_column_types, always true for sync_all_columns
* update logging and integration test on sync
* update integration tests
* test fix SF integration tests
Co-authored-by: Matt Winkler <matt.winkler@fishtownanalytics.com >
* rename integration test folder
* Update core/dbt/include/global_project/macros/materializations/incremental/incremental.sql
Accept Jeremy's suggested change
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
* Update changelog [skip ci]
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-07-21 15:49:19 -04:00
leahwicz
9f716b31b3
Moving unit tests into separate workflow ( #3588 )
...
* Moving unit tests into separate workflow
* Fixing CircleCI error
2021-07-21 12:35:04 -04:00
Kyle Wigley
3dd486d8fa
Source freshness task node selection and cli command parity ( #3554 )
...
* cli: add selection args for source freshness command
* rename command to `source freshness` and maintain alias to old command
* update and add tests for source freshness command and node selection
* update changelog, add comments
* fix formatting
* update changelog
2021-07-21 10:31:40 -04:00
Jeremy Cohen
33217891ca
Refactor relationships test to support where config ( #3583 )
...
* Rewrite relationships with CTEs
* Update changelog PR num [skip ci]
2021-07-20 19:28:09 -04:00
dependabot[bot]
1d37c4e555
Update snowflake-connector-python[secure-local-storage] requirement ( #3594 )
...
Updates the requirements on [snowflake-connector-python[secure-local-storage]](https://github.com/snowflakedb/snowflake-connector-python ) to permit the latest version.
- [Release notes](https://github.com/snowflakedb/snowflake-connector-python/releases )
- [Commits](https://github.com/snowflakedb/snowflake-connector-python/commits )
---
updated-dependencies:
- dependency-name: snowflake-connector-python[secure-local-storage]
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-07-20 14:01:35 -04:00
Nathaniel May
9f62ec2153
add rust performance runner
2021-07-20 09:42:12 -04:00
Jeremy Cohen
372eca76b8
Bump werkzeug lower bound, werkzeug-refresh-token script ( #3590 )
...
* Update werkzeug, refresh-token script
* Add changelog note
2021-07-20 08:45:56 -04:00
Ian Knox
e3cb050bbc
Merge pull request #3490 from dbt-labs/feature/dbt-build
...
dbt `build`
2021-07-19 19:09:50 -05:00
Jeremy Cohen
0ae93c7f54
Fix store_failures modifier for unique, not_null ( #3577 )
...
* Fix store_failures modifier for unique, not_null
* Add test, changelog
2021-07-16 14:50:06 -04:00
leahwicz
1f6386d760
Adding missing v0.20.0 files back to develop ( #3566 )
2021-07-15 16:46:22 -04:00
Ian Knox
66eb3964e2
PR feedback, Changelog
2021-07-14 13:23:46 -05:00
Nathaniel May
f460d275ba
add experimental parser tracking ( #3553 )
2021-07-09 17:13:34 -04:00
Jeremy Cohen
fb91bad800
Update create_adapter_plugins.py ( #3509 )
...
* Update create_adapter_plugins.py
* Bump, changelog [skip ci]
2021-07-09 14:53:01 -04:00
Jeremy Cohen
eaec22ae53
Reconcile changelogs bw 0.20.latest + develop ( #3552 )
2021-07-09 13:39:42 -04:00
Jeremy Cohen
b7c1768cca
Update changelog ( #3551 )
2021-07-09 13:35:16 -04:00
Nathaniel May
387b26a202
Merge pull request #3549 from dbt-labs/test-tweak
...
Tweak function to return a value instead of asserting
2021-07-09 12:20:22 -04:00
Ian Knox
8a1e6438f1
Revert downstream blocking on test failure logic due to test issues 😧
2021-07-09 10:52:12 -05:00
Ian Knox
aaac5ff2e6
New node discovery logic for internal work queue
2021-07-09 10:52:12 -05:00
Ian Knox
4dc29630b5
Race condition fix + downstream test blocking
2021-07-09 10:52:12 -05:00
Ian Knox
f716631439
Grouped Topologic sorting logic
2021-07-09 10:52:12 -05:00
Ian Knox
648a780850
First pass dbt build, cli only
2021-07-09 10:51:53 -05:00
Jeremy Cohen
de0919ff88
Include dbt-docs changes for 0.20.0 final ( #3544 )
2021-07-09 11:33:13 -04:00
Nathaniel May
8b1ea5fb6c
return value from test function
2021-07-08 18:14:38 -04:00
Jeremy Cohen
85627aafcd
Speed up Snowflake column comments, while still avoiding errors ( #3543 )
...
* Have our cake and eat it quickly, too
* Update changelog
2021-07-07 18:18:26 -04:00
Jeremy Cohen
49065158f5
Add gitignore, ignore pycache ( #3536 )
2021-07-06 11:51:17 -04:00
Gerda Shank
bdb3049218
Merge pull request #3522 from dbt-labs/pp_fix_simul_model_patch_deletion
...
Partial parsing: check if a node has already been deleted [#3516 ]
2021-07-06 11:46:02 -04:00
jmriego
e10d1b0f86
'+' config prefix handling whitespace ( #3526 )
...
* '+' config prefix handling whitespace
* rerun ci
2021-07-02 12:25:18 -04:00
Drew Banin
83b98c8ebf
Merge pull request #3499 from dbt-labs/fix/agate-undesirable-casting
...
Prevent Agate from coercing values in query result sets
2021-07-01 10:53:28 -04:00
Jeremy Cohen
b9d5123aa3
Update changelog for 0.20.0rc2
2021-06-30 16:24:25 -04:00
Gerda Shank
c09300bfd2
Partial parsing: check if a node has already been deleted [ #3516 ]
2021-06-30 11:35:13 -04:00
leahwicz
fc490cee7b
Adding link to plugin release instructions
2021-06-30 09:51:53 -04:00
Jeremy Cohen
3baa3d7fe8
Update badge links
2021-06-30 08:46:33 -04:00
Jeremy Cohen
764c7c0fdc
Update logo, badges ( #3518 )
...
* Fix logo, badges
* Update commit hash
* A few more edits
* Point Circle badge to develop [skip ci]
* Final fixups [skip ci]
2021-06-30 08:43:05 -04:00
Drew Banin
c97ebbbf35
Update License.md ( #3517 )
2021-06-29 22:43:50 -04:00
Drew Banin
85fe32bd08
Move to 0.21 changelog section
2021-06-29 20:21:31 -04:00
Drew Banin
eba3fd2255
Merge branch 'develop' of github.com:fishtown-analytics/dbt into fix/agate-undesirable-casting
2021-06-29 20:20:18 -04:00
Jeremy Cohen
e2f2c07873
Include dbt-docs changes for 0.20.0rc2 ( #3511 )
2021-06-29 18:44:42 -04:00
Nathaniel May
70850cd362
Merge pull request #3497 from fishtown-analytics/experimental-parser-rust
...
Experimental Parser: Swap python extractor for rust dependency
2021-06-29 16:21:26 -04:00
Gerda Shank
16992e6391
Merge pull request #3505 from fishtown-analytics/pp_testing
...
Expand partial parsing tests; fix macro partial parsing [#3449 ]
2021-06-29 15:45:33 -04:00
Gerda Shank
fd0d95140e
Expand partial parsing tests; fix macro partial parsing [ #3449 ]
2021-06-29 11:53:38 -04:00
Nathaniel May
ac65fcd557
update experimental parser to use rust dependency
2021-06-28 17:30:07 -04:00
Kyle Wigley
4d246567b9
Update project load tracking to include experimental parser info ( #3495 )
...
* Fix docs generation for cross-db sources in REDSHIFT RA3 node (#3408 )
* Fix docs generating for cross-db sources
* Code reorganization
* Code adjustments according to flake8
* Error message adjusted to be more precise
* CHANGELOG update
* add static analysis info to parsing data
* update changelog
* don't use `meta`! need better separation between dbt internal objects and external facing data. hacked an internal field on the manifest to save off this parsing info for the time being
* fix partial parsing case
Co-authored-by: kostek-pl <67253952+kostek-pl@users.noreply.github.com >
2021-06-28 10:09:50 -04:00
Drew Banin
1ad1c834f3
( #2984 ) Prevent Agate from coercing values in query result sets
2021-06-26 13:24:30 -04:00
Jeremy Cohen
41610b822c
Touch up init, fix missing adapter errors ( #3483 )
...
* Some love to init
* Update changelog
2021-06-24 15:51:36 -04:00
leahwicz
c794600242
Move starter project into dbt repo ( #3474 )
...
Addresses issue #3005
2021-06-22 11:03:01 -04:00
Jessica Laughlin
9d414f6ec3
add optional SSL parameters to Postgres connector ( #3473 )
...
Allows users to optionally set values for sslcert, sslkey, and
sslrootcert in their Postgres profiles.
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-06-21 17:52:59 -04:00
Ted Conbeer
552e831306
Feature/faster materializations with fewer transactions ( #3468 )
...
* drop if exists in view and table materializations
* Add integration test
* Add changelog entry
* PR Review 1
2021-06-21 14:58:32 -04:00
leahwicz
c712c96a0b
Commenting our flaky integration test ( #3477 )
...
Test continually fails on time out so commenting out until we can fix it
2021-06-21 10:51:36 -04:00
Gerda Shank
eb46bfc3d6
Merge pull request #3460 from fishtown-analytics/minimal_pp_validation
...
Add minimal validation of schema file yaml prior to partial parsing
2021-06-18 15:08:59 -04:00
dependabot[bot]
f52537b606
Update typing-extensions requirement in /core ( #3310 )
...
Updates the requirements on [typing-extensions](https://github.com/python/typing ) to permit the latest version.
- [Release notes](https://github.com/python/typing/releases )
- [Commits](https://github.com/python/typing/compare/3.7.4...3.10.0.0 )
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com >
2021-06-18 13:05:55 -04:00
dependabot[bot]
762419d2fe
Update werkzeug requirement from <2.0,>=0.15 to >=0.15,<3.0 in /core ( #3390 )
...
Updates the requirements on [werkzeug](https://github.com/pallets/werkzeug ) to permit the latest version.
- [Release notes](https://github.com/pallets/werkzeug/releases )
- [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst )
- [Commits](https://github.com/pallets/werkzeug/compare/0.15.0...2.0.1 )
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-06-18 10:16:28 -04:00
dependabot[bot]
4feb7cb15b
Update idna requirement from <3,>=2.5 to >=2.5,<4 in /core ( #3429 )
...
Updates the requirements on [idna](https://github.com/kjd/idna ) to permit the latest version.
- [Release notes](https://github.com/kjd/idna/releases )
- [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.rst )
- [Commits](https://github.com/kjd/idna/compare/v2.5...v3.2 )
---
updated-dependencies:
- dependency-name: idna
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-06-18 09:52:38 -04:00
Gerda Shank
eb47b85148
Add minimal validation of schema file yaml prior to partial parsing
...
[#3246 ]
2021-06-17 13:25:04 -04:00
Anders
9faa019a07
dispatch logic of new test materialization ( #3461 )
...
* dispatch logic of new test materialization
allow custom adapters to override the core test select statement functionality
* rename macro
* raconte moi une histoire
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-06-17 12:09:34 -04:00
Jeremy Cohen
9589dc91fa
Fix quoting for stringy test configs ( #3459 )
...
* Fix quoting for stringy test configs
* Update changelog
2021-06-16 14:26:07 -04:00
Gerda Shank
14507a283e
Merge pull request #3454 from fishtown-analytics/adapter_dispatch_infinite_recursion
...
Fix macro depends_on recursion error when macros call themselves (dbt…utils.datediff)
2021-06-11 11:22:05 -04:00
Gerda Shank
af0fe120ec
Fix macro depends_on recursion error when macros call themselves (dbt_utils.datediff)
2021-06-11 10:28:16 -04:00
Gerda Shank
16501ec1c6
Merge pull request #3445 from fishtown-analytics/fix_serialization
...
create _lock when deserializing manifest, plus cleanup file serialization
2021-06-11 09:30:10 -04:00
leahwicz
bf867f6aff
Add minor version release template ( #3442 )
...
Add minor version release template
2021-06-09 09:31:21 -04:00
kostek-pl
eb4ad4444f
Fix docs generation for cross-db sources in REDSHIFT RA3 node ( #3408 )
...
* Fix docs generating for cross-db sources
* Code reorganization
* Code adjustments according to flake8
* Error message adjusted to be more precise
* CHANGELOG update
2021-06-09 09:08:52 -04:00
Gerda Shank
8fdba17ac6
create _lock when deserializing manifest, plus cleanup file
...
serialization
2021-06-08 16:08:12 -04:00
Github Build Bot
abe8e83945
Merge remote-tracking branch 'origin/releases/0.20.0rc1' into develop
2021-06-04 19:02:42 +00:00
Github Build Bot
02cbae1f9f
Release dbt v0.20.0rc1
2021-06-04 18:31:00 +00:00
Gerda Shank
65908b395f
Merge pull request #3432 from fishtown-analytics/docs_references
...
Save doc file node references and use in partial parsing
2021-06-04 13:50:52 -04:00
Gerda Shank
4971395d5d
Save doc file node references and use in partial parsing
2021-06-04 13:26:19 -04:00
Kyle Wigley
eeec2038aa
bump run results and manifest artifact schemas versions ( #3421 )
...
* bump run results and manifest artifact schemas versions
* update changelog
2021-06-04 12:32:24 -04:00
Kyle Wigley
4fac086556
Add deprecation warning for providing packages to adapter.dispatch ( #3420 )
...
* add deprecation warning
* update changelog
* update deprecation message
* fix flake8
2021-06-04 10:11:31 -04:00
Jeremy Cohen
8818061d59
One more dbt-docs change for v0.20.0rc1 ( #3430 )
...
* Include dbt-docs changes for 0.20.0rc1
* One more dbt-docs change for 0.20.0rc1
2021-06-04 08:54:45 -04:00
leahwicz
b195778eb9
Updating triggers to not have duplicate runs ( #3417 )
...
Updating triggers to not have duplicate runs
2021-06-04 08:38:46 -04:00
Jeremy Cohen
de1763618a
Avoid repeating nesting in test query ( #3427 )
2021-06-03 16:57:17 -04:00
Jeremy Cohen
7485066ed4
Include dbt-docs changes for 0.20.0rc1 ( #3424 )
2021-06-03 16:43:16 -04:00
Jeremy Cohen
15ce956380
Call out breaking changes in changelog [skip ci] ( #3418 )
2021-06-03 13:50:34 -04:00
Gerda Shank
e5c63884e2
Merge pull request #3364 from fishtown-analytics/move_partial_parsing
...
Move partial parsing to end of parsing. Switch to using msgpack for saved manifest.
2021-06-02 16:02:10 -04:00
Kyle Wigley
9fef62d83e
update test subquery alias ( #3414 )
...
* update subquery alias
* update changelog
2021-06-02 15:44:17 -04:00
Gerda Shank
7563b997c2
Move partial parsing to end of parsing and implement new partial parsing
...
method. Switch to using msgpack for saved manifest.
2021-06-02 15:32:22 -04:00
leahwicz
291ff3600b
Creating test workflow in Actions ( #3396 )
...
Moving the Windows tests to Actions and adding Mac tests as well
2021-06-02 11:54:37 -04:00
Kyle Wigley
2c405304ee
New test configs ( #3392 )
...
* New test configs: where, limit, warn_if, error_if
* update test task and tests
* fix flake8
* Update core/dbt/parser/schemas.py
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
* Update core/dbt/parser/schema_test_builders.py
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
* respond to some feedback
* add failures field
* add failures to results
* more feedback
* add some test coverage
* dev review feedback
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-06-02 10:28:55 -04:00
Jeremy Cohen
1e5a7878e5
Hard code dispatch namespaces for fivetran_utils? ( #3403 )
...
* Hard code for fivetran_utils
* Add changelog entry [skip cii]
2021-06-01 10:58:18 -04:00
Stephen Bailey
d89e1d7f85
Add "tags" and "meta" properties to exposure schema ( #3405 )
...
* Add meta and tags to parsed and unparsed
* Fix tests
* Add meta and tags to unparsed schema
* Update Changelog
* Remove "optional" specifier and add default values
* Fix exposure schemas for PG Integration tests
2021-06-01 08:19:10 -04:00
Jeremy Cohen
98c015b775
Fix statically extracting macro calls for macro.depends_on.macros to be ( #3363 )
...
used in parsing schema tests by looking at the arguments to
adapter.dispatch. Includes providing an alternative way of specifying
macro search order in project config.
Collaboratively developed with Jeremy Cohen.
Co-authored-by: Gerda Shank <gerda@fishtownanalytics.com >
2021-05-27 17:13:47 -04:00
Ian Knox
a56502688f
Merge pull request #3384 from fishtown-analytics/feature/ls_in_RPC
...
Add `ls` to RPC server
2021-05-27 14:42:49 -05:00
Jeremy Cohen
c0d757ab19
dbt test --store-failures ( #3316 )
2021-05-27 15:21:29 -04:00
Ian Knox
e68fd6eb7f
PR Feedback
2021-05-27 12:25:18 -05:00
Ian Knox
90edc38859
fix ls test
2021-05-26 11:59:34 -05:00
matt-winkler
0f018ea5dd
Bugfix/issue 3350 snowflake non json response ( #3365 )
...
* attempt at solving with while loop
* added comment on loop
* update changelog
* modified per drew's suggestions
* updates after linting
2021-05-26 12:18:16 -04:00
Ian Knox
1be6254363
updated changelog
2021-05-26 10:39:23 -05:00
Ian Knox
760af71ed2
Merge branch 'feature/ls_in_RPC' of github.com:fishtown-analytics/dbt into feature/ls_in_RPC
2021-05-26 10:21:30 -05:00
Ian Knox
82f5e9f5b2
added additional fields to list response (unique_id, original_file_path)
2021-05-26 10:21:12 -05:00
Ian Knox
988c187db3
Merge branch 'develop' into feature/ls_in_RPC
2021-05-26 09:40:17 -05:00
Ian Knox
b23129982c
update default threading settings for all tasks
2021-05-26 09:37:49 -05:00
Daniel Mateus Pires
4d5d0e2150
🔨 Add ssh-client + update git using debian backports in Docker image ( #3338 )
...
* 🔨 Add ssh-client and update git version using debian backports in Docker image
* ✏️ Update CHANGELOG
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-05-24 18:39:32 -04:00
Jon Natkins
c0c487bf77
Running a snapshot with missing required configurations results in un... ( #3385 )
...
* Running a snapshot with missing required configurations results in uncaught Python exception (#3381 )
* Running a snapshot with missing required configurations results in uncaught Python exception
* Add fix details to CHANGELOG
* Update CHANGELOG.md
* Update invalid snapshot test with new/improved error message
* Improve changelog message and contributors addition
2021-05-24 15:48:08 -04:00
Nathaniel May
835d805079
Merge pull request #3374 from fishtown-analytics/experiment/dbt_jinja
...
Add experimental parser behind flag
2021-05-21 18:25:08 -04:00
Nathaniel May
c2a767184c
add changelog entry
2021-05-21 17:02:40 -04:00
Nathaniel May
1e7c8802eb
add --use-experimental-parser flag, depend on tree-sitter-jinja2, add unit tests
2021-05-21 17:01:55 -04:00
Nathaniel May
a76ec42586
static parsing skateboard
2021-05-21 17:01:55 -04:00
PJGaetan
7418f36932
Allow to use a costum field as check-cols updated_at ( #3376 )
...
* Allow to use a costum field as check-cols updated_at
* Clarify changlog /w jtcohen6
2021-05-21 16:15:56 -04:00
Ian Knox
f9ef5e7e8e
changelog
2021-05-21 09:27:54 -05:00
Ian Knox
dbfa351395
tests and fixes
2021-05-21 09:22:18 -05:00
Jeremy Cohen
e775f2b38e
Use shutil.which to find executable path ( #3299 )
...
* (explicitly) find the executable before running run_cmd
#3035
* fix undefined var
* use Executable to say exe not found and use full pth to exe
* changelog for #3035
* Nest shutil.which for better error msg
Co-authored-by: Majid alDosari <majidaldosari-github@yahoo.com >
Co-authored-by: Kyle Wigley <kyle@fishtownanalytics.com >
2021-05-20 17:30:56 -04:00
dependabot[bot]
6f27454be4
Bump jinja2 from 2.11.2 to 2.11.3 in /core ( #3077 )
...
Bumps [jinja2](https://github.com/pallets/jinja ) from 2.11.2 to 2.11.3.
- [Release notes](https://github.com/pallets/jinja/releases )
- [Changelog](https://github.com/pallets/jinja/blob/master/CHANGES.rst )
- [Commits](https://github.com/pallets/jinja/compare/2.11.2...2.11.3 )
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-05-20 10:54:07 -04:00
Ian Knox
201723d506
ls in RPC works
2021-05-20 08:27:16 -05:00
Josh Devlin
17555faaca
Add a better error for undefined macros ( #3343 )
...
* Add a better error for undefined macros
* Add check/error when installed packages < specified packages
* fix integration tests
* Fix issue with null packages
* Don't call _get_project_directories() twice
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
* Fix some integration and unit tests
* Make mypy happy
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
* Fix docs and rpc integration tests
* Fix (almost) all the rpc tests
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2021-05-19 14:08:30 -04:00
Kyle Wigley
36e0ab9f42
Merge pull request #3339 from fishtown-analytics/fix/packaging-dep
...
add 'packaging' package to dbt-core
2021-05-19 11:24:30 -04:00
Kyle Wigley
6017bd6cba
Merge branch 'develop' into fix/packaging-dep
2021-05-19 10:26:06 -04:00
Claire Carroll
30fed8d421
Refactor schema tests ( #3367 )
2021-05-18 18:15:13 -04:00
Kyle Wigley
8ac5cdd2e1
Merge pull request #3351 from fishtown-analytics/fix/bigquery-debug-task
...
Fix debug task for BigQuery connections
2021-05-14 14:22:23 -04:00
Kyle Wigley
114ac0793a
update changelog
2021-05-14 11:02:04 -04:00
Kyle Wigley
d0b750461a
fix debug task for bigquery connections
2021-05-14 11:01:23 -04:00
Jeremy Cohen
9693170eb9
Cleanup v0.20.0 changelog [skip ci] ( #3323 )
2021-05-13 18:44:41 -04:00
Jeremy Cohen
bbab6c2361
Separate compiled_path in manifest + printer ( #3327 )
2021-05-13 18:29:17 -04:00
Gerda Shank
cfe3636c78
Merge pull request #3342 from fishtown-analytics/split_out_schema_parsing
...
Do schema file parsing after parsing all other files
2021-05-13 16:11:05 -04:00
Gerda Shank
aadf3c702e
Do schema file parsing after parsing all other files
2021-05-13 15:26:03 -04:00
Kyle Wigley
1eac726a07
fix expected redshift stats (I think the possible values for the svv_table_info.encoded column changed)
2021-05-13 13:49:02 -04:00
Gerda Shank
85e2c89794
Merge pull request #3345 from fishtown-analytics/package_name_macros
...
Handle macros with package names in schema test rendering
2021-05-12 21:53:46 -04:00
Eli Kastelein
fffcd3b404
Check if a snowflake column exists before altering its comment ( #3149 )
...
* Check if column exists when altering column comments in snowflake
* Add new test class for persist docs models with missing columns
* Parallel run all integration tests after unit (#3328 )
* don't clobber default args
* update changelog
* Update changelog for PR #3149
* Pull in upstream changes
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
Co-authored-by: Kyle Wigley <kyle@fishtownanalytics.com >
2021-05-12 21:33:01 -04:00
peiwangdb
fbfef4b1a3
fix integration test failure due to timezone ( #3344 )
...
* fix integration test failure due to timezone
* update changelog
2021-05-12 21:28:19 -04:00
Gerda Shank
526a6c0d0c
Merge branch 'develop' into package_name_macros
2021-05-12 21:21:45 -04:00
Ian Knox
1f33b6a74a
Merge pull request #3335 from fishtown-analytics/feature/schema_tests_are_more_unique
...
Feature/schema tests are more unique
2021-05-12 19:18:50 -05:00
Ian Knox
95fc6d43e7
Additional PR feedback
2021-05-12 13:29:24 -05:00
Kyle Wigley
d8c261ffcf
Merge pull request #3340 from fishtown-analytics/fix/default-test-args
...
Stop clobbering default args for test definitions
2021-05-12 10:25:53 -04:00
Kyle Wigley
66ea0a9e0f
update changelog
2021-05-12 09:54:20 -04:00
Gerda Shank
435b542e7b
Add macros with project name to schema test context, and recursively get
...
macros
2021-05-12 09:51:44 -04:00
Ian Knox
10cd06f515
PR feedback
2021-05-12 08:38:50 -05:00
Jeremy Cohen
9da1868c3b
Parallel run all integration tests after unit ( #3328 )
2021-05-11 13:23:38 -04:00
Kyle Wigley
2649fac4a4
don't clobber default args
2021-05-11 10:45:16 -04:00
Kyle Wigley
6e05226e3b
update changelog
2021-05-11 07:18:15 -04:00
Kyle Wigley
c1c3397f66
add 'packaging' package to dbt-core
2021-05-11 07:07:12 -04:00
Ian Knox
2065db2383
Testing changes to support hashes
2021-05-10 15:49:50 -05:00
Ian Knox
08fb868b63
Adds hash to generic test unique_ids
2021-05-10 15:45:07 -05:00
Aram Panasenco
8d39ef16b6
Now generating a run_results.json even when no nodes are selected ( #3315 )
...
* Now generating a run_results.json even when no nodes are selected.
* Typo in changelog
* Modified changelog in accordance with @jtcohen6's suggestions. Also fixed @TeddyCr's entry.
2021-05-06 08:07:54 -04:00
Teddy
66c5082aa7
Feature/3117 dbt deps timeout ( #3275 )
...
* added logic to timeout request after 60 seconds + unit test
* fixed typo in comment
* Update test URL for azure CI failure
* Update changelog + change endpoint for timeout test + added retry logic to the timeout exception
* updated exception catching to a generic one + adjusted tests based on error catching + renamed test file accordingly
* updated comment in test_registry_get_request_exception.py to reflect new approach
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-05-05 23:40:21 -04:00
Kyle Wigley
26fb58bd1b
Merge pull request #3318 from fishtown-analytics/fix/ephemeral-compilation
...
[moving fix to current release] Fix ephemeral model compilation
2021-05-04 14:54:19 -04:00
Kyle Wigley
fed8826043
update changelog
2021-05-04 10:02:56 -04:00
Kyle Wigley
9af78a3249
fix tests after merge
2021-05-04 08:57:31 -04:00
Kyle Wigley
bf1ad6cd17
Merge pull request #3139 from fishtown-analytics/fix/ephemeral-compile-sql
...
Fix compiled sql for ephemeral models
2021-05-04 08:39:39 -04:00
Github Build Bot
15e995f2f5
Merge remote-tracking branch 'origin/releases/0.20.0b1' into develop
2021-05-03 11:33:09 +00:00
Github Build Bot
b3e73b0de8
Release dbt v0.20.0b1
2021-05-03 11:08:53 +00:00
Jeremy Cohen
dd2633dfcb
Include parent adapters in dispatch ( #3296 )
...
* Add test, expect fail
* Include parent adapters in dispatch
* Use adapter type, not credentials type
* Adjust adapter_macro deprecation test
* fix test/unit/test_context.py to use postgres profile
* Add changelog note
* Redshift default column encoding now AUTO
Co-authored-by: Gerda Shank <gerda@fishtownanalytics.com >
2021-05-02 18:01:57 -04:00
Karthik Ramanathan
29f0278451
prevent locks in incremental full refresh ( #2998 )
2021-05-02 17:19:25 -04:00
Angel Montana
f0f98be692
Support dots in model names: Make FQN model selectors work with them ( #3247 )
...
* Support dots in model names. They are useful as namespace separators
* Update CHANGELOG.md
* Update contributor list
* Extend integration test to support models whose name contains dots
* Cleanup fqn selection login
* Explain condition check
* Support dots in model names: integration tests
* Make linter happy: remove trailing whitespaces
* Support dots in model names: integration test for seeds
* revert 66c26facbd . Integration tests to support dots in model names implemented in 007_graph_selection_tests
* Test model with dots only in postgres. It's meant to work in combination with custom macros for other databases
* Support dot as namespace separators: integration test
* Support dots in test names: integration test
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-04-28 17:32:14 -04:00
Kyle Wigley
5956a64b01
Merge pull request #3305 from fishtown-analytics/fix/bigquery-no-project
...
add unit test and move default logic to mashumaro hook
2021-04-28 14:52:53 -04:00
Daniel Mateus Pires
5fb36e3e2a
Issue 275: Support dbt package dependencies in Git subdirectories ( #3267 )
...
* 🔨 Extend git package contract and signatures to pass `subdirectory`
* Add sparse checkout logic
* ✅ Add test
* 🧹 Lint
* ✏️ Update CHANGELOG
* 🐛 Make os.path.join safe
* Use a test-container with an updated `git` version
* 🔨 Fix integration tests
* 📖 Update CHANGELOG contributors to include this PR
* 🧪 Parameterize the test
* Use new test-container published by @kwigley (contains more recent version of git)
* Use repositories managed by fishtown
* 🧘♂️ Merge the CHANGELOG
* 🤦♂️ Remove repetition of my contribution on the CHANGELOG
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-04-28 09:40:05 -04:00
Cor
9d295a1d91
Add TestCLIInvocationWithProfilesAndProjectDir ( #3176 )
...
* Add TestCLIInvocationWithProfilesAndProjectDir
* Add context manager for changing working directory
* Add fixture for changing working directory to temporary directory
* Add missing import
* Move custom profile into function
* Add function to create profiles directory
* Remove path and union
* Make temporary_working_directory a context manager
* Fix some issues, wrong naming and invocation
* Update test with profiles and project dir
* Use a fixture for the sub command
* Run test for debug and run sub command
* Resolve inheritance
* Remove profiles from project dir
* Use pytest mark directly instead of use_profile
* Use parameterize on class
* Remove parameterize
* Create separate test for each subcommand
* Set profiles_dir to false
* Add run test
* Use abspath
* Add entry to change log
* Add suggested changes
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
* Remove test_postgres_run_with_profiles_separate_from_project_dir
* Add JCZuurmond to change log
* Add test_postgres_run_with_profiles_separate_from_project_dir
* Remove HEAD
* Fix wrong merge
* Add deps before test
* Add run to test which runs test command
* Sort tests
* Force rerun
* Force rerun
* Force rerun
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2021-04-28 08:20:02 -04:00
Jeremy Cohen
39f350fe89
Be less greedy in test selection expansion ( #3235 )
...
* Expand test selection iff all first-order parents are selected
* Renumber new integration test
* PR feedback
* Fix flake
2021-04-27 15:52:16 -04:00
Kyle Wigley
8c55e744b8
Merge pull request #3286 from fishtown-analytics/feature/schema-test-materialization
...
use test materialization for schema/generic tests
2021-04-27 08:05:42 -04:00
Jeremy Cohen
a260d4e25b
Merge pull request #3106 from arzavj/postgres_create_indexes
...
Postgres: ability to create indexes
2021-04-27 08:02:26 -04:00
Jeremy Cohen
509797588f
Merge branch 'develop' into postgres_create_indexes
2021-04-27 07:28:38 -04:00
Kyle Wigley
2eed20f1f3
update changelog
2021-04-23 09:51:56 -04:00
Kyle Wigley
1d7b4c0db2
update integration tests, fix tests for bigquery
2021-04-22 16:24:13 -04:00
Kyle Wigley
ac8cd788cb
use test materialization for schema/generic tests, update integration tests
2021-04-22 11:26:13 -04:00
Gerda Shank
33dc970859
Merge pull request #3272 from fishtown-analytics/test_context_regression
...
Add necessary macros to schema test context namespace
2021-04-20 12:36:40 -04:00
Kyle Wigley
f73202734c
Merge pull request #3261 from fishtown-analytics/feature/test-jinja-block
...
Add `test` Jinja tag
2021-04-19 09:39:53 -04:00
Jeremy Cohen
32bacdab4b
Merge pull request #3270 from dmateusp/dmateusp/3268/dbt_deps_support_commit_hashes
...
Issue 3268: Support commit hashes in dbt deps
2021-04-18 18:17:27 -04:00
Daniel Mateus Pires
6113c3b533
📖 Add myself to contribs
2021-04-18 21:33:01 +01:00
Gerda Shank
1c634af489
Add necessary macros to schema test context namespace [ #3229 ] [ #3240 ]
2021-04-16 13:21:18 -04:00
Daniel Mateus Pires
428cdea2dc
✏️ Update CHANGELOG
2021-04-16 10:58:04 +01:00
Daniel Mateus Pires
f14b55f839
✅ Add test
2021-04-16 10:54:35 +01:00
Daniel Mateus Pires
5934d263b8
✨ Support git commit as revision
2021-04-16 10:21:49 +01:00
Kyle Wigley
3860d919e6
use ternary and f-strings
2021-04-15 14:49:07 -04:00
Kyle Wigley
fd0b9434ae
update changelog
2021-04-15 14:49:07 -04:00
Kyle Wigley
efb30d0262
first pass at adding test jinja block
2021-04-15 14:48:12 -04:00
Jeremy Cohen
cee0bfbfa2
Merge pull request #3257 from fishtown-analytics/feature/test-config-parity
...
Feature: test config parity
2021-04-15 14:15:19 -04:00
Jeremy Cohen
dc684d31d3
Add changelog entry
2021-04-15 14:04:34 -04:00
Gerda Shank
bfdf7f01b5
Merge pull request #3248 from fishtown-analytics/load_all_files
...
Preload all project files at start of parsing [#3244 ]
2021-04-14 13:33:32 -04:00
Gerda Shank
2cc0579b6e
Preload all project files at start of parsing [ #3244 ]
2021-04-14 09:57:26 -04:00
Jeremy Cohen
bfc472dc0f
Cleanup + integration tests
2021-04-13 20:04:53 -04:00
Jeremy Cohen
ea4e3680ab
Configure tests from dbt_project.yml
2021-04-13 20:04:42 -04:00
Jeremy Cohen
f02139956d
Add support for disabling schema tests
2021-04-13 20:04:42 -04:00
Arzav Jain
cacbd1c212
Updated Changelog.md
2021-04-12 09:38:42 -07:00
Arzav Jain
3f78bb7819
more tests
2021-04-12 09:37:54 -07:00
Arzav Jain
aa65b01fe3
basic tests for table materialization
2021-04-12 09:37:54 -07:00
Arzav Jain
4f0968d678
fix style to get unit tests to pass
2021-04-12 09:37:54 -07:00
Arzav Jain
118973cf79
respond to pr comments
2021-04-12 09:37:54 -07:00
Arzav Jain
df7cc0521f
remove no longer used truncate_string macro
2021-04-12 09:37:54 -07:00
Arzav Jain
40c02d2cc9
Respond to pr comments; approach inspired by persist_docs()
2021-04-12 09:37:54 -07:00
Arzav Jain
be70b1a0c1
first pass
2021-04-12 09:37:44 -07:00
Kyle Wigley
7ec5c122e1
Merge pull request #3228 from fishtown-analytics/cleanup/makefile
...
Better make targets
2021-04-12 10:49:59 -04:00
Jeremy Cohen
a10ab99efc
Merge pull request #3243 from fuchsst/fix/3241-add-missing-exposures-property
...
Fix/3241 add missing exposures property
2021-04-12 09:27:40 -04:00
Fuchs, Stefan
9f4398c557
added contribution to changelog
2021-04-11 18:26:29 +02:00
Fuchs, Stefan
d60f6bc89b
added exposures property to manifest
2021-04-11 18:26:05 +02:00
Kyle Wigley
617eeb4ff7
test code changes without reinstalling everything
2021-04-08 12:07:55 -04:00
Kyle Wigley
5b55825638
add flaky logic to bigquery
2021-04-07 09:02:44 -04:00
Kyle Wigley
103d524db5
update changelog
2021-04-06 14:33:05 -04:00
Kyle Wigley
babd084a9b
better make targets, some descriptions
2021-04-06 14:33:05 -04:00
Gerda Shank
749f87397e
Merge pull request #3219 from fishtown-analytics/partial_parsing
...
Use Manifest instead of ParseResult [#3163 ]
2021-04-06 14:05:17 -04:00
Gerda Shank
307d47ebaf
Use Manifest instead of ParseResults [ #3163 ]
2021-04-06 13:51:43 -04:00
Jeremy Cohen
6acd4b91c1
Merge pull request #3227 from fishtown-analytics/update/changelog-0191
...
Update changelog for 0.19.1, 0.18.2
2021-04-05 16:36:28 -04:00
Jeremy Cohen
f4a9530894
Update changelog per 0.18.2
2021-04-05 15:11:08 -04:00
Jeremy Cohen
ab65385a16
Update changelog per 0.19.1
2021-04-05 15:08:33 -04:00
Jeremy Cohen
ebd761e3dc
Merge pull request #3156 from max-sixty/patch-1
...
Update google cloud dependencies
2021-04-02 15:13:24 -04:00
Maximilian Roos
3b942ec790
Merge branch 'develop' into patch-1
2021-04-02 10:09:26 -07:00
Maximilian Roos
b373486908
Update CHANGELOG.md
2021-04-02 10:08:51 -07:00
Maximilian Roos
c8cd5502f6
Update setup.py
2021-04-02 10:05:16 -07:00
Maximilian Roos
d6dd968c4f
Pin to major versions
2021-03-31 18:51:53 -07:00
Jeremy Cohen
b8d73d2197
Merge pull request #3182 from cgopalan/app-name-for-postgres
...
Set application_name for Postgres connections
2021-03-31 15:56:59 -04:00
Kyle Wigley
17e57f1e0b
Merge pull request #3181 from fishtown-analytics/feature/data-test-materialization
...
Adding test materialization, implement for data tests
2021-03-30 14:21:43 -04:00
Kyle Wigley
e21bf9fbc7
code comments and explicit function call
2021-03-30 12:47:40 -04:00
Kyle Wigley
12e281f076
update changelog
2021-03-29 09:50:37 -04:00
Kyle Wigley
a5ce658755
first pass using materialization for data tests
2021-03-29 09:49:02 -04:00
Kyle Wigley
ce30dfa82d
Merge pull request #3204 from fishtown-analytics/updates/tox
...
dev env clean up and improvements
2021-03-29 09:47:49 -04:00
Kyle Wigley
c04d1e9d5c
fix circleci tests (forcing azure pipeline tests)
2021-03-29 09:19:05 -04:00
Kyle Wigley
80031d122c
fix windows rpc tests
2021-03-29 09:19:05 -04:00
Kyle Wigley
943b090c90
debug windows tests
2021-03-29 09:19:04 -04:00
Kyle Wigley
39fd53d1f9
fix typos, set max-line-length (force azure pipeline tests)
2021-03-29 09:19:04 -04:00
Kyle Wigley
777e7b3b6d
update changelog
2021-03-29 09:19:04 -04:00
Kyle Wigley
2783fe2a9f
fix last CI step
2021-03-29 09:11:06 -04:00
Kyle Wigley
f5880cb001
CI tweaks
2021-03-29 09:11:06 -04:00
Kyle Wigley
26e501008a
use new docker image for tests
2021-03-29 09:11:06 -04:00
Kyle Wigley
2c67e3f5c7
update tox, update makefile, run tests natively by default, general dev workflow cleanup
2021-03-29 09:11:06 -04:00
Kyle Wigley
033596021d
Merge pull request #3148 from fishtown-analytics/dependabot/pip/plugins/snowflake/snowflake-connector-python-secure-local-storage--2.4.1
...
Bump snowflake-connector-python[secure-local-storage] from 2.3.6 to 2.4.1 in /plugins/snowflake
2021-03-29 09:09:49 -04:00
Kyle Wigley
f36c72e085
update changelog
2021-03-29 08:36:33 -04:00
Kyle Wigley
fefaf7b4be
update snowflake deps
2021-03-26 16:04:48 -04:00
Chandrakant Gopalan
91431401ad
Updated changelog
2021-03-26 16:01:47 -04:00
Chandrakant Gopalan
59d96c08a1
Add tests for application_name
2021-03-26 15:25:38 -04:00
Chandrakant Gopalan
f10447395b
Fix tests
2021-03-25 21:18:42 -04:00
Chandrakant Gopalan
c2b6222798
Merge branch 'develop' of https://github.com/fishtown-analytics/dbt into app-name-for-postgres
2021-03-25 21:03:40 -04:00
Chandrakant Gopalan
3a58c49184
Default application_name to dbt
2021-03-25 21:03:08 -04:00
Jeremy Cohen
440a5e49e2
Merge pull request #3041 from yu-iskw/issue-3040
...
Pass the default scopes to the default BigQuery credentials
2021-03-24 17:59:32 -04:00
Jeremy Cohen
77c10713a3
Merge pull request #3100 from prratek/specify-cols-to-update
...
Gets columns to update from config for BQ and Snowflake
2021-03-22 17:55:49 +01:00
Jeremy Cohen
48e367ce2f
Merge branch 'develop' into specify-cols-to-update
2021-03-22 13:56:11 +01:00
Jeremy Cohen
934c23bf39
Merge pull request #3145 from jmcarp/jmcarp/bigquery-job-labels
...
Parse query comment and use as bigquery job labels.
2021-03-22 13:30:08 +01:00
Chandrakant Gopalan
e0febcb6c3
Set application_name for Postgres connections
2021-03-20 13:24:44 -04:00
Joshua Carp
044a6c6ea4
Cleanups from code review.
2021-03-20 00:59:55 -04:00
Prratek Ramchandani
8ebbc10572
Merge branch 'develop' into specify-cols-to-update
2021-03-18 09:55:12 -04:00
Jeremy Cohen
7435828082
Merge pull request #3165 from cgopalan/dup-macro-message
...
Raise proper error message if duplicate macros found
2021-03-17 14:53:50 +01:00
Jeremy Cohen
369b595e8a
Merge branch 'develop' into dup-macro-message
2021-03-17 12:46:38 +01:00
Jeremy Cohen
9a6d30f03d
Merge pull request #3158 from techytushar/fix#3147
...
Feature to add _n alias to same column names #3147
2021-03-17 12:15:35 +01:00
Prratek Ramchandani
6bdd01d52b
Merge branch 'develop' into specify-cols-to-update
2021-03-16 16:13:30 -04:00
prratek
bae9767498
add my name to contributors
2021-03-16 16:07:50 -04:00
prratek
b0e50dedb8
update changelog
2021-03-16 15:59:34 -04:00
Prratek Ramchandani
96bfb3b259
Update core/dbt/include/global_project/macros/materializations/common/merge.sql
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2021-03-16 15:54:04 -04:00
Prratek Ramchandani
909068dfa8
leave quoting for merge_update_columns to the user
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2021-03-16 15:53:25 -04:00
Chandrakant Gopalan
f4c74968be
Add to contributor list
2021-03-16 13:53:51 -04:00
Chandrakant Gopalan
0e958f3704
Add to changelog
2021-03-16 13:51:42 -04:00
Chandrakant Gopalan
a8b2942f93
Fix duplicate macro path message
2021-03-16 13:40:18 -04:00
Tushar Mittal
564fe62400
Add unit tests for SQL process_results
2021-03-16 21:42:36 +05:30
Chandrakant Gopalan
5c5013191b
Fix failing test
2021-03-14 17:46:53 -04:00
Chandrakant Gopalan
31989b85d1
Fix flake8 errors
2021-03-14 15:46:47 -04:00
Chandrakant Gopalan
5ed4af2372
Raise proper error message if duplicate macros found
2021-03-14 15:33:15 -04:00
prratek
4d18e391aa
correct the load date for updated entries in "update seed"
2021-03-13 12:59:41 -05:00
prratek
2feeb5b927
Merge remote-tracking branch 'origin/specify-cols-to-update' into specify-cols-to-update
2021-03-13 12:26:22 -05:00
Prratek Ramchandani
2853f07875
use correct config var name
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2021-03-13 12:26:12 -05:00
prratek
4e6adc07a1
switch to correct data dir for second run
2021-03-13 12:25:02 -05:00
Jeremy Cohen
6a5ed4f418
Merge pull request #3161 from fishtown-analytics/pin/agate-1.6.2
...
Pin agate<1.6.2 to fix installation
2021-03-12 17:11:40 +01:00
Jeremy Cohen
ef25698d3d
Pin agate>=1.6,<1.6.2 to fix installation
2021-03-12 12:17:53 +01:00
Tushar Mittal
429dcc7000
Update changelog
2021-03-11 22:10:57 +05:30
Gerda Shank
ab3f994626
Merge pull request #3157 from fishtown-analytics/track_resource_counts
...
Track resource counts
2021-03-11 09:42:09 -05:00
Tushar Mittal
5f8235fcfc
Feature to add _n alias to same column names #3147
...
Signed-off-by: Tushar Mittal <chiragmittal.mittal@gmail.com >
2021-03-11 19:44:58 +05:30
Gerda Shank
db325d0fde
Track resource counts
2021-03-10 23:07:02 -05:00
Maximilian Roos
8dc1f49ac7
Update google cloud dependencies
2021-03-10 19:57:57 -08:00
Joshua Carp
9fe2b651ed
Merge branch 'develop' of github.com:fishtown-analytics/dbt into jmcarp/bigquery-job-labels
2021-03-09 23:32:11 -05:00
Jeremy Cohen
24e4b75c35
Merge pull request #3151 from bastienboutonnet/chore/bump_hologram_and_dataclasses
...
chore: allow dataclasses >= 0.6, < 0.9
2021-03-09 17:59:29 +01:00
Bastien Boutonnet
34174abf26
add changelog entry
2021-03-09 15:56:39 +01:00
Bastien Boutonnet
af778312cb
relax dependency on dataclasses requirements as in hologram
2021-03-09 15:55:22 +01:00
Bastien Boutonnet
280f5614ef
bump hologram to 0.0.14
2021-03-08 22:24:46 +01:00
Joshua Carp
8566a46793
Add BigQuery job labels to changelog.
2021-03-08 15:50:00 -05:00
Joshua Carp
af3c3f4cbe
Add tests for bigquery label sanitize helper.
2021-03-08 15:43:53 -05:00
Bastien Boutonnet
034a44e625
fix git install link
2021-03-08 21:02:00 +01:00
Bastien Boutonnet
84155fdff7
point hologram install to my fork and up dataclasses
2021-03-08 20:42:16 +01:00
prratek
8255c913a3
change test logic for new seed directories
2021-03-06 19:12:33 -05:00
prratek
4d4d17669b
refactor seeds directory structure and names
2021-03-06 19:08:23 -05:00
prratek
540a0422f5
modify seeds to contain load date and some modified records
2021-03-06 19:06:35 -05:00
prratek
de4d7d6273
Revert "modify some records and the expected result"
...
This reverts commit 1345d955
2021-03-06 18:39:35 -05:00
prratek
1345d95589
modify some records and the expected result
2021-03-06 18:30:50 -05:00
prratek
a5bc19dd69
paste in some data for seeds
2021-03-06 18:27:02 -05:00
prratek
25b143c8cc
WIP test case and empty seeds
2021-03-06 18:17:07 -05:00
Joshua Carp
82cca959e4
Merge branch 'develop' of github.com:fishtown-analytics/dbt into jmcarp/bigquery-job-labels
2021-03-05 09:39:48 -05:00
dependabot[bot]
d52374a0b6
Bump snowflake-connector-python[secure-local-storage]
...
Bumps [snowflake-connector-python[secure-local-storage]](https://github.com/snowflakedb/snowflake-connector-python ) from 2.3.6 to 2.4.1.
- [Release notes](https://github.com/snowflakedb/snowflake-connector-python/releases )
- [Commits](https://github.com/snowflakedb/snowflake-connector-python/compare/v2.3.6...v2.4.1 )
Signed-off-by: dependabot[bot] <support@github.com >
2021-03-05 06:15:17 +00:00
Joshua Carp
c71a18ca07
Hyphenate query comment fields and fix deserialization bug.
2021-03-05 00:09:17 -05:00
Joshua Carp
8d73ae2cc0
Address comments from code review.
2021-03-04 10:20:15 -05:00
Joshua Carp
7b0c74ca3e
Fix lint.
2021-03-04 00:34:46 -05:00
Joshua Carp
62be9f9064
Sanitize bigquery labels.
2021-03-04 00:14:50 -05:00
Joshua Carp
2fdc113d93
Parse query comment and use as bigquery job labels.
2021-03-04 00:06:59 -05:00
Gerda Shank
b70fb543f5
Merge pull request #3138 from fishtown-analytics/mashumaro-cleanup
...
Use updated Mashumaro code
2021-03-03 15:46:56 -05:00
Gerda Shank
31c88f9f5a
Use updated Mashumaro code
2021-03-03 13:39:51 -05:00
Prratek Ramchandani
af3a818f12
loop over column_name instead of column.name
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2021-03-03 09:27:05 -05:00
prratek
a07532d4c7
revert changes to incremental materializations
2021-03-02 22:14:58 -05:00
prratek
fb449ca4bc
rename new config var to merge_update_columns
2021-03-02 22:11:12 -05:00
prratek
4da65643c0
use merge_update_columns when getting merge sql
2021-03-02 22:10:09 -05:00
Prratek Ramchandani
bf64db474c
fix typo
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2021-03-02 21:59:21 -05:00
Kyle Wigley
344a14416d
Merge pull request #3065 from fishtown-analytics/feature/write-artifact-schema
...
Collect and write json schema for dbt artifacts
2021-02-22 12:47:35 -05:00
Jeremy Cohen
be47a0c5db
Merge pull request #3117 from fishtown-analytics/fix/deps-git-warn-if-main-master
...
git package: warn if specified revision master, main
2021-02-22 10:49:31 +01:00
prratek
808b980301
move "update cols" incremental test to snowflake models
2021-02-20 14:40:55 -05:00
prratek
3528480562
test incremental model w/ subset of cols to update
2021-02-19 23:17:00 -05:00
prratek
6bd263d23f
add incremental_update_columns to Snowflake & BQ config schemas
2021-02-19 21:17:39 -05:00
prratek
2b9aa3864b
rename config field to incremental_update_columns
2021-02-19 21:13:05 -05:00
Prratek Ramchandani
81155caf88
use get_columns_in_relation as default for snowflake
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2021-02-19 21:10:34 -05:00
Jeremy Cohen
c7c057483d
sry flake8
2021-02-19 15:19:03 +01:00
Jeremy Cohen
7f5170ae4d
Warn if main or master is specified, too
2021-02-19 11:19:36 +01:00
Jeremy Cohen
49b8693b11
Merge pull request #3104 from VasiliiSurov/develop
...
3057 Moving from 'master' to 'HEAD' default branch in git
2021-02-18 20:29:34 +01:00
Jeremy Cohen
d7b0a14eb5
Merge branch 'develop' into develop
2021-02-18 19:55:54 +01:00
Jeremy Cohen
8996cb1e18
Merge pull request #2976 from pcasteran/fix/2940-bq-incremental-var-declaration
...
Fix `_dbt_max_partition` declaration and initialization for BigQuery incremental models
2021-02-18 18:54:16 +01:00
Jeremy Cohen
38f278cce0
Merge pull request #3111 from fishtown-analytics/feature/architecture-md
...
Add ARCHITECTURE.md
2021-02-18 14:55:05 +01:00
Jeremy Cohen
bb4e475044
Add aARCHITECTURE.md [skip ci]
2021-02-17 14:58:43 +01:00
Pascal Casteran
4fbe36a8e9
Moved changelog entry up to v0.20.0
2021-02-16 19:12:22 +01:00
Pascal Casteran
a1a40b562a
Updated CHANGELOG.md
2021-02-16 19:09:22 +01:00
Pascal Casteran
3a4a1bb005
Fix _dbt_max_partition declaration and initialization for BigQuery incremental models
2021-02-16 19:09:22 +01:00
Prratek Ramchandani
4f8c10c1aa
default to get_columns_in_relation if not specified in config
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2021-02-15 20:58:39 -05:00
VS
4833348769
Merge remote-tracking branch 'origin/develop' into develop
2021-02-15 17:06:43 -05:00
VS
ad07d59a78
3057 Changelog.md
2021-02-15 17:06:36 -05:00
VasiliiSurov
e8aaabd1d3
Merge pull request #1 from VasiliiSurov/3057_dbt_deps
...
3057 Replacing 'master' to 'HEAD' for default git commit
2021-02-15 17:02:13 -05:00
Kyle Wigley
d7d7396eeb
update dependabot config, turn off rebasing
2021-02-15 14:42:44 -05:00
Kyle Wigley
41538860cd
Revert "Merge branch '0.19.latest' into develop"
...
This reverts commit 5c9f8a0cf0 , reversing
changes made to 36d1bddc5b .
2021-02-15 13:26:18 -05:00
Kyle Wigley
5c9f8a0cf0
Merge branch '0.19.latest' into develop
2021-02-15 12:11:52 -05:00
Github Build Bot
11c997c3e9
Merge remote-tracking branch 'origin/releases/0.19.1b2' into 0.19.latest
2021-02-15 17:04:12 +00:00
Github Build Bot
1b1184a5e1
Release dbt v0.19.1b2
2021-02-15 17:00:50 +00:00
Kyle Wigley
4ffcc43ed9
Merge pull request #3101 from fishtown-analytics/vendor-mashumaro
...
Vendor mashumaro code with dbt specific modifications
2021-02-15 10:36:10 -05:00
Kyle Wigley
4ccaac46a6
Update core/mashumaro/types.py
...
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-02-15 10:02:50 -05:00
Kyle Wigley
ba88b84055
Update core/mashumaro/serializer/base/dict.py
...
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2021-02-15 10:02:45 -05:00
prratek
9086634c8f
get columns to update from config for BQ and Snowflake
2021-02-13 11:09:25 -05:00
VS
e88f1f1edb
3057 Replacing 'master' to 'HEAD' for default git commit
2021-02-12 22:34:49 -05:00
Kyle Wigley
13c7486f0e
add license info
2021-02-12 17:32:15 -05:00
Kyle Wigley
8e811ba141
vendor mashumaro code in dbt, include as another module in dbt-core
2021-02-12 17:19:28 -05:00
Github Build Bot
c5d86afed6
Merge remote-tracking branch 'origin/releases/0.19.1b1' into 0.19.latest
2021-02-12 16:10:31 +00:00
Github Build Bot
43a0cfbee1
Release dbt v0.19.1b1
2021-02-12 16:06:52 +00:00
Jeremy Cohen
8567d5f302
Fix int64, ts partitions. Rework tests
2021-02-12 15:58:47 +01:00
Jeremy Cohen
36d1bddc5b
Merge pull request #3098 from fishtown-analytics/fix/bq-insert-overwrite-int-ts-partitions
...
[BQ] Fix insert_overwrite with int + ts partitions
2021-02-12 15:56:59 +01:00
Kyle Wigley
bf992680af
Merge pull request #3090 from fishtown-analytics/fix/dbt-mashumaro-dep
...
Move dbt-mashumaro dep to setup.py to prep for beta release
2021-02-12 09:32:31 -05:00
Kyle Wigley
e064298dfc
move dbt-mashumaro dep to setup.py to prep for beta release
2021-02-12 08:51:14 -05:00
Jeremy Cohen
e01a10ced5
Fix int64, ts partitions. Rework tests
2021-02-12 14:03:03 +01:00
Gerda Shank
2aa10fb1ed
Use version 0.0.13 of Hologram
2021-02-11 11:48:58 -05:00
Gerda Shank
66f442ad76
Merge pull request #3071 from fishtown-analytics/hologram-version-0.0.13
...
Use version 0.0.13 of Hologram
2021-02-11 11:46:57 -05:00
Kyle Wigley
11f1ecebcf
update changelog
2021-02-11 11:21:40 -05:00
Kyle Wigley
e339cb27f6
first pass at writing out artifact schema to destination
2021-02-11 11:20:38 -05:00
Kyle Wigley
bce3232b39
Merge pull request #3062 from fishtown-analytics/feature/dependabot-config
...
Add dependabot config
2021-02-11 10:22:13 -05:00
Kyle Wigley
b08970ce39
Merge pull request #3069 from fishtown-analytics/fix/update-docs
...
Update docs with permalinks and new git branching strategy
2021-02-11 10:17:57 -05:00
Gerda Shank
533f88ceaf
Use version 0.0.13 of Hologram
2021-02-10 14:08:27 -05:00
Jeremy Cohen
c8f0469a44
Merge pull request #3036 from ran-eh/patch-2
...
Fix incorrect error message (trivial)
2021-02-10 12:00:43 +01:00
Kyle Wigley
a1fc24e532
update docs with permalinks and new git branching
2021-02-10 00:31:40 -05:00
Ran Ever-Hadani
d80daa48df
Fix flake8 error
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2021-02-09 14:07:50 -08:00
Kyle Wigley
92aae2803f
update changelog
2021-02-09 09:53:32 -05:00
Gerda Shank
77cbbbfaf2
Performance fixes, including supporting libyaml, caching
...
mapped_fields in the classes for 'from_dict', removing deepcopy
on fqn_search, separating validation from 'from_dict',
and special handling for dbt internal not_null and unique tests.
Use TestMacroNamespace instead of original in order to limit
the number of macros in the context. Integrate mashumaro into
dbt to improve performance of 'from_dict' and 'to_dict'
2021-02-08 14:06:14 -05:00
Gerda Shank
6c6649f912
Performance fixes, including supporting libyaml, caching
...
mapped_fields in the classes for 'from_dict', removing deepcopy
on fqn_search, separating validation from 'from_dict',
and special handling for dbt internal not_null and unique tests.
Use TestMacroNamespace instead of original in order to limit
the number of macros in the context. Integrate mashumaro into
dbt to improve performance of 'from_dict' and 'to_dict'
2021-02-05 15:23:55 -05:00
Yu ISHIKAWA
55fbaabfda
Pass the default scopes to the default BigQuery credentials
2021-01-29 18:09:05 +09:00
Ran Ever-Hadani
56c2518936
Correct message to be more general
...
as per https://github.com/fishtown-analytics/dbt/pull/3036#pullrequestreview-578328868
2021-01-28 11:22:14 -08:00
Kyle Wigley
2b48152da6
Merge branch 'dev/0.19.1' into dev/margaret-mead
2021-01-27 17:16:13 -05:00
Christophe Blefari
e743e23d6b
Update CHANGELOG to release fix in dbt 0.19.1 version
2021-01-27 16:57:29 -05:00
Christophe Blefari
f846f921f2
Bump werkzeug upper bound dependency constraint to include version 1.0
2021-01-27 16:55:56 -05:00
Ran Ever-Hadani
e52a599be6
Add fix
2021-01-27 13:52:01 -08:00
Ran Ever-Hadani
99744bd318
Fix incorrect error message (trivial)
2021-01-27 13:09:32 -08:00
Github Build Bot
1060035838
Merge remote-tracking branch 'origin/releases/0.19.0' into dev/kiyoshi-kuromiya
2021-01-27 18:02:37 +00:00
Github Build Bot
69cc20013e
Release dbt v0.19.0
2021-01-27 17:39:48 +00:00
Github Build Bot
3572bfd37d
Merge remote-tracking branch 'origin/releases/0.19.0rc3' into dev/kiyoshi-kuromiya
2021-01-27 16:42:46 +00:00
Github Build Bot
a6b82990f5
Release dbt v0.19.0rc3
2021-01-27 16:07:41 +00:00
Kyle Wigley
540c1fd9c6
Merge pull request #3019 from fishtown-analytics/fix/cleanup-dockerfile
...
Clean up docker resources
2021-01-25 10:19:45 -05:00
Jeremy Cohen
46d36cd412
Merge pull request #3028 from NiallRees/lowercase_cte_names
...
Make generated CTE test names lowercase to match style guide
2021-01-25 14:39:26 +01:00
NiallRees
a170764fc5
Add to contributors
2021-01-25 11:16:00 +00:00
NiallRees
f72873a1ce
Update CHANGELOG.md
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2021-01-25 11:13:32 +00:00
NiallRees
82496c30b1
Changelog
2021-01-24 16:35:40 +00:00
NiallRees
cb3c007acd
Make generated CTE test names lowercase to match style guide
2021-01-24 16:19:20 +00:00
Jeremy Cohen
cb460a797c
Merge pull request #3018 from lynxcare/fix-issue-debug-exit-code
...
dbt debug should return 1 when one of the tests fail
2021-01-21 16:36:03 +01:00
Kyle Wigley
1b666d01cf
add a dependabot config, hopefully this ignores docker/requirements/*.txt files
2021-01-21 09:51:35 -05:00
Sam Debruyn
df24c7d2f8
Merge branch 'dev/margaret-mead' into fix-issue-debug-exit-code
2021-01-21 15:39:18 +01:00
Sam Debruyn
133c15c0e2
move in changelog to v0.20
2021-01-21 15:38:31 +01:00
Kyle Wigley
116e18a19e
rename testing dockerfile
2021-01-21 09:28:17 -05:00
Sam Debruyn
ec0af7c97b
remove exitcodes and sys.exit
2021-01-21 10:36:05 +01:00
Jeremy Cohen
a34a877737
Merge pull request #2974 from rvacaru/fix-bug-2731
...
Fix bug #2731 on stripping query comments for snowflake
2021-01-21 09:54:22 +01:00
Sam Debruyn
f018794465
fix flake test - formatting
2021-01-20 21:09:58 +01:00
Sam Debruyn
d45f5e9791
add missing conditions
2021-01-20 18:15:32 +01:00
Razvan Vacaru
04bd0d834c
added extra unit test
2021-01-20 18:06:17 +01:00
Sam Debruyn
ed4f0c4713
formatting
2021-01-20 18:04:21 +01:00
Sam Debruyn
c747068d4a
use sys.exit
2021-01-20 16:51:06 +01:00
Kyle Wigley
aa0fbdc993
update changelog
2021-01-20 10:33:18 -05:00
Kyle Wigley
b50bfa7277
- rm older dockerfiles
...
- add dockerfile from dbt-releases
- rename the development dockerfile to Dockerfile.dev to avoid confusion
2021-01-20 10:23:03 -05:00
Sam Debruyn
e91988f679
use ExitCodes enum for exit code
2021-01-20 16:09:41 +01:00
Sam Debruyn
3ed1fce3fb
update changelog
2021-01-20 16:06:24 +01:00
Sam Debruyn
e3ea0b511a
dbt debug should return 1 when one of the tests fail
2021-01-20 16:00:58 +01:00
Razvan Vacaru
c411c663de
moved unit tests and updated changelog.md
2021-01-19 19:04:58 +01:00
Razvan Vacaru
1c6f66fc14
Merge branch 'dev/margaret-mead' of https://github.com/fishtown-analytics/dbt into fix-bug-2731
2021-01-19 19:01:01 +01:00
Jeremy Cohen
1f927a374c
Merge pull request #2928 from yu-iskw/issue-1843
...
Support require_partition_filter and partition_expiration_days in BQ
2021-01-19 12:11:39 +01:00
Jeremy Cohen
07c4225aa8
Merge branch 'dev/margaret-mead' into issue-1843
2021-01-19 11:24:59 +01:00
Github Build Bot
42a85ac39f
Merge remote-tracking branch 'origin/releases/0.19.0rc2' into dev/kiyoshi-kuromiya
2021-01-14 17:41:49 +00:00
Github Build Bot
16e6d31ee3
Release dbt v0.19.0rc2
2021-01-14 17:21:25 +00:00
Kyle Wigley
a6db5b436d
Merge pull request #2996 from fishtown-analytics/fix/rm-ellipses
...
Remove ellipses printed while parsing
2021-01-14 10:39:16 -05:00
Kyle Wigley
47675f2e28
update changelog
2021-01-14 09:28:28 -05:00
Kyle Wigley
0642bbefa7
remove ellipses printed while parsing
2021-01-14 09:28:05 -05:00
Kyle Wigley
43da603d52
Merge pull request #3009 from fishtown-analytics/fix/exposure-parsing
...
Fix exposure parsing to allow other resources with the same name
2021-01-14 09:26:02 -05:00
Kyle Wigley
f9e1f4d111
update changelog
2021-01-13 11:54:20 -05:00
Jeremy Cohen
1508564e10
Merge pull request #3008 from fishtown-analytics/feat/print-exposure-stats-too
...
Add exposures to print_compile_stats
2021-01-13 15:58:13 +01:00
Kyle Wigley
c14e6f4dcc
add test for dupe exposures and dupe model/exposure name
2021-01-13 08:55:22 -05:00
Jeremy Cohen
75b6a20134
Add exposures to Found list
2021-01-12 19:07:52 +01:00
Kyle Wigley
d82a07c221
tweak exposure parsing logic
2021-01-12 12:41:51 -05:00
Jeremy Cohen
c6f7dbcaa5
Merge pull request #3006 from stpierre/postgres-unpin-botocore
...
postgres: Don't pin botocore version
2021-01-12 13:59:55 +01:00
Chris St. Pierre
82cd099e48
Update CHANGELOG
2021-01-12 06:20:09 -06:00
Chris St. Pierre
546c011dd8
postgres: Don't pin botocore version
...
`snowflake-connector-python` doesn't pin it, and it restricts us to a
much older version of boto3 than the boto3 pin would otherwise allow
(specifically, botocore<1.15 requires boto3<1.12).
2021-01-11 17:25:03 -06:00
Jeremy Cohen
10b33ccaf6
Merge pull request #3004 from mikaelene/Snapshot_merge_WHEN_MATCHED
...
This changes makes the macro easier to read and workable on SQL Server
2021-01-11 16:42:09 +01:00
mikaelene
bc01572176
Sane as #3003 . But for postgres
2021-01-11 16:04:38 +01:00
mikaelene
ccd2064722
This changes makes the macro easier to read and makes the code work for SQL Server without a custom adapter macro. Solved #3003
2021-01-11 15:04:23 +01:00
mikaelene
0fb42901dd
This changes makes the macro easier to read and makes the code work for SQL Server without a custom adapter macro. Solved #3003
2021-01-11 14:58:07 +01:00
Jeremy Cohen
a4280d7457
Merge pull request #3000 from swanderz/tsql_not_equal_workaround
...
Tsql not equal workaround
2021-01-11 09:40:33 +01:00
Anders
6966ede68b
Update CHANGELOG.md
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2021-01-10 20:54:37 -08:00
Anders
27dd14a5a2
Update core/dbt/include/global_project/macros/materializations/snapshot/strategies.sql
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2021-01-10 20:54:10 -08:00
Anders
2494301f1e
Update CHANGELOG.md
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2021-01-10 20:53:52 -08:00
Anders Swanson
f13143accb
for posterity
2021-01-08 13:23:13 -08:00
Anders Swanson
26d340a917
temp hack
2021-01-08 12:14:08 -08:00
Anders Swanson
cc75cd4102
no tsql support for condA != condB
2021-01-08 12:10:15 -08:00
Anders Swanson
cf8615b231
Merge branch 'dev/kiyoshi-kuromiya' of https://github.com/fishtown-analytics/dbt into dev/kiyoshi-kuromiya
2021-01-08 12:03:15 -08:00
Jeremy Cohen
30f473a2b1
Merge pull request #2994 from fishtown-analytics/copyedit-changelog
...
Light cleanup of v0.19.0 changelogs
2021-01-07 16:00:02 +01:00
Jeremy Cohen
4618709baa
Lightly edit v0.19 changelogs
2021-01-07 10:13:43 +01:00
Razvan Vacaru
16b098ea42
updated CHANGELOG.md
2021-01-04 17:43:03 +01:00
Razvan Vacaru
b31c4d407a
Fix #2731 stripping snowflake comments in multiline queries
2021-01-04 17:41:00 +01:00
Kyle Wigley
28c36cc5e2
Merge pull request #2988 from fishtown-analytics/fix/dockerfile
...
Manually fix requirements for dockerfile using new pip version
2021-01-04 09:10:05 -05:00
Kyle Wigley
6bfbcb842e
manually fix dockerfile using new pip version
2020-12-31 13:53:50 -05:00
Github Build Bot
a0eade4fdd
Merge remote-tracking branch 'origin/releases/0.19.0rc1' into dev/kiyoshi-kuromiya
2020-12-29 23:07:35 +00:00
Github Build Bot
ee24b7e88a
Release dbt v0.19.0rc1
2020-12-29 22:52:34 +00:00
Anders Swanson
c9baddf9a4
Merge branch 'master' of https://github.com/fishtown-analytics/dbt into dev/kiyoshi-kuromiya
2020-12-22 23:11:09 -08:00
Kyle Wigley
c5c780a685
Merge pull request #2972 from fishtown-analytics/feature/update-dbt-docs
...
dbt-docs changes for v0.19.0-rc1
2020-12-22 14:07:20 -05:00
Kyle Wigley
421aaabf62
Merge pull request #2961 from fishtown-analytics/feature/add-adapter-query-stats
...
Include adapter response info in execution results
2020-12-22 13:57:07 -05:00
Kyle Wigley
86788f034f
update changelog
2020-12-22 13:30:50 -05:00
Kyle Wigley
232d3758cf
update dbt docs
2020-12-22 13:17:51 -05:00
Kyle Wigley
71bcf9b31d
update changelog
2020-12-22 13:08:12 -05:00
Kyle Wigley
bf4ee4f064
update api, fix tests, add placeholder for test/source results
2020-12-22 12:13:37 -05:00
Kyle Wigley
aa3bdfeb17
update naming
2020-12-21 13:35:15 -05:00
Jeremy Cohen
ce6967d396
Merge pull request #2966 from fishtown-analytics/fix/add-ctes-comment
...
Update comments for _add_ctes()
2020-12-18 10:54:37 -05:00
Yu ISHIKAWA
330065f5e0
Add a condition for require_partition_filter
2020-12-18 11:14:03 +09:00
Yu ISHIKAWA
944db82553
Remove unnecessary code for print debug
2020-12-18 11:14:03 +09:00
Yu ISHIKAWA
c257361f05
Fix syntax
2020-12-18 11:14:03 +09:00
Yu ISHIKAWA
ffdbfb018a
Implement tests in test_bigquery_changing_partitions.py
2020-12-18 11:14:01 +09:00
Yu ISHIKAWA
cfa2bd6b08
Remove tests fromm test_bigquery_adapter_specific.py
2020-12-18 11:13:16 +09:00
Yu ISHIKAWA
51e90c3ce0
Format
2020-12-18 11:13:16 +09:00
Yu ISHIKAWA
d69149f43e
Update
2020-12-18 11:13:15 +09:00
Yu ISHIKAWA
f261663f3d
Add debug code
2020-12-18 11:13:15 +09:00
Yu ISHIKAWA
e5948dd1d3
Update
2020-12-18 11:13:15 +09:00
Yu ISHIKAWA
5f13aab7d8
Print debug
2020-12-18 11:13:15 +09:00
Yu ISHIKAWA
292d489592
Format code
2020-12-18 11:13:15 +09:00
Yu ISHIKAWA
0a01f20e35
Update CHANGELOG.md
2020-12-18 11:13:11 +09:00
Yu ISHIKAWA
2bd08d5c4c
Support require_partition_filter and partition_expiration_days in BQ
2020-12-18 11:12:47 +09:00
Jeremy Cohen
adae5126db
Merge pull request #2954 from fishtown-analytics/feature/defer-tests
...
Feature: defer tests
2020-12-17 18:01:14 -05:00
Kyle Wigley
dddf1bcb76
first pass at adding query stats, naming tbd
2020-12-17 16:39:02 -05:00
Jeremy Cohen
d23d4b0fd4
Merge pull request #2963 from tyang209/issue-2931
...
Bumped boto3 version uppper range for dbt-redshift
2020-12-17 14:30:47 -05:00
Tao Yang
658f7550b3
Merge branch 'dev/kiyoshi-kuromiya' into issue-2931
2020-12-17 08:58:49 -08:00
Kyle Wigley
cfb50ae21e
Merge pull request #2960 from fishtown-analytics/feature/python-39
...
Test python3.9
2020-12-17 11:11:56 -05:00
Jeremy Cohen
9b0a365822
Update comments for _add_ctes()
2020-12-17 10:35:04 -05:00
Jeremy Cohen
97ab130619
Merge pull request #2958 from fishtown-analytics/fix/keyerror-defer-missing-parent
...
Fix KeyError from defer + deletion
2020-12-17 10:29:51 -05:00
Tao Yang
3578fde290
Bumped boto3 version uppper range for dbt-redshift
2020-12-16 20:03:53 -08:00
Jeremy Cohen
f382da69b8
Changelog
2020-12-16 17:46:00 -05:00
Jeremy Cohen
2da3d215c6
Add test case to repro bug
2020-12-16 17:38:27 -05:00
Kyle Wigley
43ed29c14c
update changelog
2020-12-16 16:29:48 -05:00
Jeremy Cohen
9df0283689
Truthier?
2020-12-16 14:55:27 -05:00
Jeremy Cohen
04b82cf4a5
What is backward may not be forward
2020-12-16 14:55:27 -05:00
Jeremy Cohen
274c3012b0
Add defer to rpc test method
2020-12-16 14:53:25 -05:00
Jeremy Cohen
2b24a4934f
defer tests, too
2020-12-16 14:42:00 -05:00
Kyle Wigley
692a423072
comment out snowflake py39 tests
2020-12-16 11:27:00 -05:00
Kyle Wigley
148f55335f
address issue with py39
2020-12-16 11:25:31 -05:00
Kyle Wigley
2f752842a1
update hologram and add new envs to tox
2020-12-16 11:25:31 -05:00
Jeremy Cohen
aff72996a1
Merge pull request #2946 from fishtown-analytics/fix/defer-if-not-exist
...
Defer iff unselected reference does not exist in current env
2020-12-16 11:22:31 -05:00
Jeremy Cohen
08e425bcf6
Handle keyerror if old node missing
2020-12-16 00:24:00 -05:00
Kyle Wigley
454ddc601a
Merge pull request #2943 from fishtown-analytics/feature/refactor-run-results
...
Clean up run results
2020-12-15 12:42:22 -05:00
Jeremy Cohen
b025f208a8
Check if relation exists before deferring
2020-12-14 22:21:43 -05:00
Kyle Wigley
b60e533b9d
fix printer output
2020-12-14 19:50:17 -05:00
Kyle Wigley
37af0e0d59
update changelog
2020-12-14 16:28:23 -05:00
Kyle Wigley
ac1de5bce9
more updates
2020-12-14 16:28:23 -05:00
Kyle Wigley
ef7ff55e07
flake8
2020-12-14 16:28:23 -05:00
Kyle Wigley
608db5b982
code cleanup + swap node with unique_id
2020-12-14 16:28:23 -05:00
Kyle Wigley
8dd69efd48
address test failures
2020-12-14 16:28:23 -05:00
Kyle Wigley
73f7fba793
fix printing test status
2020-12-14 16:28:23 -05:00
Kyle Wigley
867e2402d2
chugging along
2020-12-14 16:28:23 -05:00
Kyle Wigley
a3b9e61967
first pass, lots of TODO's [skip ci]
2020-12-14 16:28:22 -05:00
Jeremy Cohen
cd149b68e8
Merge pull request #2920 from joellabes/2913-docs-block-exposures
...
Render docs blocks in exposures
2020-12-13 18:38:23 -05:00
Joel Labes
cd3583c736
Merge branch 'dev/kiyoshi-kuromiya' into 2913-docs-block-exposures
2020-12-13 14:27:37 +13:00
Joel Labes
441f86f3ed
Add test.notebook_info to expected manifest
2020-12-13 14:25:37 +13:00
Joel Labes
f62bea65a1
Move model.test.view_summary to parent map instead of child map
2020-12-13 14:11:04 +13:00
Jeremy Cohen
886b574987
Merge pull request #2939 from fishtown-analytics/fix/big-seed-smaller-path
...
Use diff file path for big seed checksum
2020-12-07 11:18:15 -05:00
Joel Labes
2888bac275
Merge branch 'dev/kiyoshi-kuromiya' into 2913-docs-block-exposures
2020-12-07 21:17:21 +13:00
Joel Labes
35c9206916
Fix test failure (?)
2020-12-07 21:15:44 +13:00
Joel Labes
c4c5b59312
Stab at updating parent and child maps
2020-12-07 17:45:12 +13:00
Jeremy Cohen
f25fb4e5ac
Use diff file path for big seed checksum
2020-12-04 17:04:27 -05:00
Jeremy Cohen
868bfec5e6
Merge pull request #2907 from max-sixty/raise
...
Remove duplicate raise
2020-12-03 14:17:58 -05:00
Jeremy Cohen
e7c242213a
Merge pull request #2908 from max-sixty/bq-default-project
...
Allow BigQuery to default on project name
2020-12-03 14:17:02 -05:00
Jeremy Cohen
862552ead4
Merge pull request #2930 from fishtown-analytics/revert-2858-dependabot/pip/docker/requirements/cryptography-3.2
...
Revert dependabot cryptography upgrade for old versions
2020-12-03 13:58:26 -05:00
Jeremy Cohen
9d90e0c167
tiny changelog fixup
2020-12-03 13:27:46 -05:00
Jeremy Cohen
a281f227cd
Revert "Bump cryptography from 2.9.2 to 3.2 in /docker/requirements"
2020-12-03 12:12:15 -05:00
Maximilian Roos
5b981278db
changelog
2020-12-02 14:59:35 -08:00
Maximilian Roos
c1091ed3d1
Merge branch 'dev/kiyoshi-kuromiya' into bq-default-project
2020-12-02 14:55:27 -08:00
Maximilian Roos
08aed63455
Formatting
2020-12-02 11:19:02 -08:00
Maximilian Roos
90a550ee4f
Update plugins/bigquery/dbt/adapters/bigquery/connections.py
...
Co-authored-by: Kyle Wigley <kwigley44@gmail.com >
2020-12-02 10:41:20 -08:00
Jeremy Cohen
34869fc2a2
Merge pull request #2922 from plotneishestvo/snowflake_connector_upgrade
...
update cryptography package and snowflake connector
2020-12-02 12:34:34 -05:00
Pavel Plotnikov
3deb10156d
Merge branch 'dev/kiyoshi-kuromiya' into snowflake_connector_upgrade
2020-12-02 12:46:02 +02:00
Maximilian Roos
8c0e84de05
Move method to module func
2020-12-01 16:19:20 -08:00
Joel Labes
23be083c39
Change models folder to ref_models folder
2020-12-02 11:59:21 +13:00
Joel Labes
217aafce39
Add line break to description, fix refs and maybe fix original_file_path
2020-12-02 11:47:29 +13:00
Joel Labes
03210c63f4
Blank instead of none description
2020-12-02 10:57:47 +13:00
Joel Labes
a90510f6f2
Ref a model that actually exists
2020-12-02 10:40:34 +13:00
Joel Labes
36d91aded6
Empty description for minimal/basic exposure object tests
2020-12-01 17:56:55 +13:00
Joel Labes
9afe8a1297
Default to empty string for ParsedExposure description
2020-12-01 17:35:42 +13:00
Maximilian Roos
1e6f272034
Add test config
2020-11-30 20:06:06 -08:00
Maximilian Roos
a1aa2f81ef
_
2020-11-30 19:30:07 -08:00
Maximilian Roos
62899ef308
_
2020-11-30 16:54:21 -08:00
Joel Labes
7f3396c002
Forgot another comma 🤦
2020-12-01 12:46:26 +13:00
Joel Labes
453bc18196
Merge branch '2913-docs-block-exposures' of https://github.com/joellabes/dbt into 2913-docs-block-exposures
2020-12-01 12:42:11 +13:00
Joel Labes
dbb6b57b76
Forgot a comma
2020-12-01 12:40:51 +13:00
Joel Labes
d7137db78c
Merge branch 'dev/kiyoshi-kuromiya' into 2913-docs-block-exposures
2020-12-01 12:34:29 +13:00
Joel Labes
5ac4f2d80b
Move description arg to be below default-free args
2020-12-01 12:33:08 +13:00
Jeremy Cohen
5ba5271da9
Merge pull request #2903 from db-magnus/bq-hourly-part
...
Hourly, monthly and yearly partitions in BigQuery
2020-11-30 09:46:36 -05:00
Pavel Plotnikov
b834e3015a
update changelog md
2020-11-30 14:46:51 +02:00
Joel Labes
c8721ded62
Code review: non-optional description, docs block tests, yaml exposure attributes
2020-11-30 20:29:47 +13:00
Magnus Fagertun
1e97372d24
Update test/unit/test_bigquery_adapter.py
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2020-11-30 07:26:36 +01:00
Magnus Fagertun
fd4e111784
Update test/unit/test_bigquery_adapter.py
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2020-11-30 00:44:25 +01:00
Magnus Fagertun
75094e7e21
Update test/unit/test_bigquery_adapter.py
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2020-11-30 00:44:15 +01:00
Joel Labes
8db2d674ed
Update CHANGELOG.md
2020-11-28 15:08:13 +13:00
Pavel Plotnikov
ffb140fab3
update cryptography package and snowflake connector
2020-11-27 16:52:13 +02:00
Joel Labes
e93543983c
Follow Jeremy's wild speculation
2020-11-27 22:45:31 +13:00
Magnus Fagertun
0d066f80ff
added test and enhancements from jtcohen6
2020-11-25 21:41:51 +01:00
Magnus Fagertun
ccca1b2016
Update plugins/bigquery/dbt/adapters/bigquery/impl.py
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2020-11-25 21:17:07 +01:00
Kyle Wigley
fec0e31a25
Merge pull request #2902 from fishtown-analytics/fix/test-selection
...
set default `materialized` for test node configs
2020-11-24 12:19:40 -05:00
Kyle Wigley
d246aa8f6d
update readme
2020-11-24 10:40:01 -05:00
Maximilian Roos
66bfba2258
flake8 seems to sometimes be applied
2020-11-23 17:39:57 -08:00
Maximilian Roos
b53b4373cb
Definet database exclusively in contracts/connection.py
2020-11-23 17:32:41 -08:00
Maximilian Roos
0810f93883
Allow BigQuery to default on project name
2020-11-23 16:58:54 -08:00
Maximilian Roos
a4e696a252
Remove duplicate raise
2020-11-23 15:34:43 -08:00
Jeremy Cohen
0951d08f52
Merge pull request #2877 from max-sixty/unlock-google-api
...
Wider google-cloud dependencies
2020-11-23 14:16:12 -05:00
Jeremy Cohen
dbf367e070
Merge branch 'dev/kiyoshi-kuromiya' into unlock-google-api
2020-11-23 11:46:07 -05:00
Magnus Fagertun
6447ba8ec8
whitespace cleanup
2020-11-22 10:00:10 +01:00
Magnus Fagertun
43e260966f
uppercase and lowercase for date partitions supported
2020-11-21 01:21:07 +01:00
Magnus Fagertun
b0e301b046
typo in _partitions_match
2020-11-21 00:40:27 +01:00
Magnus Fagertun
c8a9ea4979
added month,year to date partitioning, granularity comparison to _partitions_match
2020-11-21 00:24:20 +01:00
Maximilian Roos
afb7fc05da
Changelog
2020-11-20 14:58:46 -08:00
Magnus Fagertun
14124ccca8
added tests for datetime and timestamp
2020-11-20 00:10:15 +01:00
Magnus Fagertun
df5022dbc3
moving granularity to render, not to break tests
2020-11-19 18:51:05 +01:00
Magnus Fagertun
015e798a31
more BQ partitioning
2020-11-19 17:42:27 +01:00
Kyle Wigley
c19125bb02
Merge pull request #2893 from fishtown-analytics/feature/track-parse-time
...
Add event tracking for project parse/load time
2020-11-19 10:30:46 -05:00
Kyle Wigley
0e6ac5baf1
can we just default materialization to test?
2020-11-19 09:27:31 -05:00
Magnus Fagertun
2c8d1b5b8c
Added hour, year, month partitioning BQ
2020-11-19 13:47:42 +01:00
Kyle Wigley
f7c0c1c21a
fix tests
2020-11-18 17:21:41 -05:00
Kyle Wigley
4edd98f7ce
update changelog
2020-11-18 16:19:58 -05:00
Kyle Wigley
df0abb7000
flake8 fixes
2020-11-18 16:19:58 -05:00
Kyle Wigley
4f93da307f
add event to track loading time
2020-11-18 16:19:58 -05:00
Gerda Shank
a8765d54aa
Merge pull request #2895 from fishtown-analytics/string_selectors
...
convert cli-style strings in selectors to normalized dictionaries
2020-11-18 15:53:23 -05:00
Gerda Shank
bb834358d4
convert cli-style strings in selectors to normalized dictionaries
...
[#2879 ]
2020-11-18 14:43:44 -05:00
Jeremy Cohen
ec0f3d22e7
Merge pull request #2892 from rsella/dev/kiyoshi-kuromiya
...
Change dbt list command to always return 0 as exit code
2020-11-17 11:12:55 -05:00
Riccardo Sella
009b75cab6
Fix changelog and edit additional failing tests
2020-11-17 16:38:28 +01:00
Riccardo Sella
d64668df1e
Change dbt list command to always return 0 as exit code
2020-11-17 14:49:24 +01:00
Gerda Shank
72e808c9a7
Merge pull request #2889 from fishtown-analytics/dbt-test-runner
...
Add scripts/dtr.py, dbt test runner. Bump hologram version.
2020-11-15 20:06:28 -05:00
Gerda Shank
96cc9223be
Add scripts/dtr.py, dbt test runner. Bump hologram version.
2020-11-13 14:34:10 -05:00
Gerda Shank
13b099fbd0
Merge pull request #2883 from fishtown-analytics/feature/2824-parse-only-command
...
Add parse command and collect parse timing info [#2824 ]
2020-11-13 10:19:19 -05:00
Gerda Shank
1a8416c297
Add parse command and collect parse timing info [ #2824 ]
2020-11-12 13:56:41 -05:00
Maximilian Roos
8538bec99e
_
2020-11-11 13:48:41 -08:00
Maximilian Roos
f983900597
google-cloud-bigquery goes to 3
2020-11-10 23:51:15 -08:00
Gerda Shank
3af02020ff
Merge pull request #2866 from fishtown-analytics/feature/2693-selectors-to-manifest
...
Save selector dictionary and write out in manifest [#2693 ][#2800 ]
2020-11-10 11:48:19 -05:00
Maximilian Roos
8c71488757
_
2020-11-10 08:38:43 -08:00
Gerda Shank
74316bf702
Save selector dictionary and write out in manifest [ #2693 ][ #2800 ]
2020-11-10 11:17:14 -05:00
Maximilian Roos
7aa8c435c9
Bump protobuf too
2020-11-09 17:36:41 -08:00
Maximilian Roos
daeb51253d
Unpin google-cloud dependencies
2020-11-09 17:18:42 -08:00
Jeremy Cohen
0ce2f41db4
Reorg #2837 in changelog
2020-11-09 09:46:08 -05:00
Jeremy Cohen
02e5a962d7
Merge pull request #2837 from franloza/feature/2647-relation-name-in-metadata
...
Store relation name in manifest's node and source objects
2020-11-09 09:44:40 -05:00
Jeremy Cohen
dcc32dc69f
Merge pull request #2850 from elexisvenator/patch-1
...
Postgres: Prevent temp relation identifiers from being too long
2020-11-09 09:32:35 -05:00
Gerda Shank
af3d6681dd
extend timeout for test/rpc
2020-11-06 17:45:35 -05:00
Gerda Shank
106968a3be
Merge pull request #2858 from fishtown-analytics/dependabot/pip/docker/requirements/cryptography-3.2
...
Bump cryptography from 2.9.2 to 3.2 in /docker/requirements
2020-11-06 15:54:49 -05:00
Ben Edwards
2cd56ca044
Update changelog
2020-11-03 20:58:01 +11:00
Ben Edwards
eff198d079
Add integration tests
2020-11-03 20:56:02 +11:00
Ben Edwards
c3b5b88cd2
Postgres: Prevent temp relation identifiers from being too long
...
Related: #2197
The currently postgres `make_temp_relation` adds a 29 character suffix to the end of the temp relation identifier (9 from default suffix and 20 from timestamp). This is a problem now that relations with more than 63 characters raise exceptions.
The fix is to shorten the suffix and also trim the base_relation identifier so that the total length is always less than 63 characters.
An exception can also be raised if the default suffix is overridden with a value that is too long.
2020-11-03 20:56:02 +11:00
Kyle Wigley
4e19e87bbc
Merge pull request #2859 from fishtown-analytics/fix/update-test-container
...
add unixodbc-dev to testing docker image
2020-10-30 09:56:39 -04:00
Kyle Wigley
6be6f6585d
update changelog
2020-10-29 16:52:09 -04:00
Kyle Wigley
d7579f0c99
add g++ and unixodbc-dev
2020-10-29 16:22:46 -04:00
Fran Lozano
b741679c9c
Add missing key to child map in expected_bigquery_complex_manifest
2020-10-29 17:25:18 +01:00
Fran Lozano
852990e967
Fix child_map in tests
2020-10-28 22:18:32 +01:00
Fran Lozano
21fd75b500
Fix parent_map object in tests
2020-10-28 19:59:36 +01:00
Fran Lozano
3e5d9010a3
Add snapshot to additional Redshift and Bigquery manifest tests
2020-10-28 19:39:04 +01:00
Fran Lozano
784616ec29
Add relation name to source object in manifest
2020-10-28 18:58:25 +01:00
Fran Lozano
6251d19946
Use is_ephemeral_model property instead of config.materialized
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2020-10-28 09:49:44 +01:00
dependabot[bot]
17b1332a2a
Bump cryptography from 2.9.2 to 3.2 in /docker/requirements
...
Bumps [cryptography](https://github.com/pyca/cryptography ) from 2.9.2 to 3.2.
- [Release notes](https://github.com/pyca/cryptography/releases )
- [Changelog](https://github.com/pyca/cryptography/blob/master/CHANGELOG.rst )
- [Commits](https://github.com/pyca/cryptography/compare/2.9.2...3.2 )
Signed-off-by: dependabot[bot] <support@github.com >
2020-10-27 22:22:14 +00:00
Jeremy Cohen
74eec3bdbe
Merge pull request #2855 from brangisom/brangisom/spectrum-filter-fix
...
Fix the filtering for external tables in the Redshift get_columns_in_relation macro
2020-10-27 15:03:59 -04:00
Fran Lozano
a9901c4ea7
Disable snapshot documentation testing for Redshift and Bigquery
2020-10-27 19:27:54 +01:00
Brandon Isom
348a2f91ee
Move a CHANGELOG entry
2020-10-27 11:17:13 -07:00
Fran Lozano
7115d862ea
Modify snapshot path for docs generation tests
2020-10-27 18:59:32 +01:00
Fran Lozano
52ed4aa631
Fix tests which are missing snapshot nodes
2020-10-27 18:45:00 +01:00
Fran Lozano
92cedf8931
Fix Flake8 style issue
2020-10-27 17:39:44 +01:00
Fran Lozano
e1097f11b5
Define relation_name only for non-ephemeral models, seeds and snapshots
2020-10-27 17:23:23 +01:00
Brandon Isom
eb34c0e46b
Add stuff to changelog per checklist
2020-10-26 20:17:03 -07:00
Brandon Isom
ee2181b371
Merge branch 'brangisom/spectrum-filter-fix' of github.com:brangisom/dbt into brangisom/spectrum-filter-fix
2020-10-26 19:44:45 -07:00
Brandon Isom
2a5d090e91
Pushes the table_schema = '{{ relation.schema }}' filter into the svv_external_columns CTE
2020-10-26 19:38:33 -07:00
Brandon Isom
857bebe819
Pushes the table_schema = '{{ relation.schema }}' clause down into the svv_external_columns CTE.
2020-10-26 18:29:47 -07:00
Jeremy Cohen
9728152768
Merge pull request #2851 from hochoy/add-python-regex
...
Folllow up: Support for python "re" module for doing regex in jinja templates
2020-10-26 16:27:17 -04:00
Wai Ho Choy
2566a85429
edit CHANGELOG.md
2020-10-26 12:21:30 -07:00
Wai Ho Choy
46b3130198
lint
2020-10-25 21:18:23 -07:00
Wai Ho Choy
8664516c8d
fix blank line linting
2020-10-25 12:03:10 -07:00
Wai Ho Choy
0733c246ea
add all exports from python re module
2020-10-25 11:31:33 -07:00
Fran Lozano
4203985e3e
Adapt expected_seeded_manifest method to Snowflake identifier quoting
2020-10-25 18:50:52 +01:00
Fran Lozano
900298bce7
Fix database name in relation_name in expected_run_results
2020-10-25 18:06:18 +01:00
Fran Lozano
09c37f508e
Adapt relation_name to expected_run_results parameters
2020-10-25 17:27:46 +01:00
Fran Lozano
c9e01bcc81
Fix quotes in relation name for Bigquery docs generate tests
2020-10-25 16:31:00 +01:00
Fran Lozano
b079545e0f
Adapt relation_name for Bigquery and Snowflake in docs generation tests
2020-10-25 15:58:04 +01:00
Fran Lozano
c3bf0f8cbf
Add relation_name to missing tests in test_docs_generate
2020-10-25 14:11:33 +01:00
Jeremy Cohen
e945bca1d9
Merge pull request #2596 from ran-eh/re-partition-metadata
...
Make partition metadata available to BigQuery users
2020-10-22 20:58:17 -04:00
Jeremy Cohen
bf5835de5e
Merge branch 'dev/kiyoshi-kuromiya' into re-partition-metadata
2020-10-22 20:18:31 -04:00
Ran Ever-Hadani
7503f0cb10
merge from dev/kiyoshi-kuromiya
2020-10-22 16:23:02 -07:00
Ran Ever-Hadani
3a751bcf9b
Update CHANGELOG.md
2020-10-22 15:53:25 -07:00
Jeremy Cohen
c31ba101d6
Add tests for get_partitions_metadata ( #1 )
...
* Add tests using get_partitions_metadata
* Readd asterisk to raw_execute
2020-10-21 16:00:10 -07:00
Jeremy Cohen
ecadc74d44
Merge pull request #2841 from feluelle/dev/kiyoshi-kuromiya
...
Respect --project-dir in dbt clean command
2020-10-21 16:01:11 -04:00
Jeremy Cohen
63d25aaf19
Update changelog to account for v0.19.0-b1 release
2020-10-21 09:47:36 -04:00
feluelle
5af82c3c05
Add test that checks if targets were successfully deleted
2020-10-21 10:27:39 +02:00
feluelle
8b4d74ed17
Add changelog entry for resolving issue #2840
2020-10-21 10:27:39 +02:00
feluelle
6a6a9064d5
Respect --project-dir in dbt clean command
2020-10-21 10:25:17 +02:00
Github Build Bot
b188a9488a
Merge remote-tracking branch 'origin/releases/0.19.0b1' into dev/kiyoshi-kuromiya
2020-10-21 00:46:30 +00:00
Github Build Bot
7c2635f65d
Release dbt v0.19.0b1
2020-10-21 00:35:44 +00:00
Jeremy Cohen
c67d0a0e1a
Readd bumpversion config header
2020-10-20 18:01:04 -04:00
Fran Lozano
7ee78e89c9
Add missing relation_name fields in doc generation test manifests
2020-10-20 19:18:38 +02:00
Fran Lozano
40370e104f
Fix wrong schema name in test and add missing relation_name in node
2020-10-20 18:48:59 +02:00
Fran Lozano
a8809baa6c
Merge branch 'dev/kiyoshi-kuromiya' into feature/2647-relation-name-in-metadata
2020-10-20 18:32:53 +02:00
Fran Lozano
244d5d2c3b
Merge remote-tracking branch 'upstream/dev/kiyoshi-kuromiya' into dev/kiyoshi-kuromiya
2020-10-20 18:26:28 +02:00
Fran Lozano
a0370a6617
Add relation_name to node object in docs generation tests
2020-10-20 18:22:32 +02:00
Jeremy Cohen
eb077fcc75
Merge pull request #2845 from fishtown-analytics/docs/0.19.0-b1
...
dbt-docs changes for dbt v0.19.0-b1
2020-10-20 09:57:11 -04:00
Jeremy Cohen
c5adc50eed
Make flake8 happy
2020-10-19 18:42:23 -04:00
Jeremy Cohen
6e71b6fd31
Include dbt-docs changes for v0.19.0-b1
2020-10-19 18:41:31 -04:00
Gerda Shank
278382589d
Merge pull request #2834 from fishtown-analytics/feature/remove_injected_sql
...
Remove injected_sql. Store non-ephemeral injected_sql in compiled_sql
2020-10-19 18:08:41 -04:00
Gerda Shank
6f0f6cf21a
Merge branch 'dev/0.18.1' into dev/kiyoshi-kuromiya
2020-10-19 11:30:52 -04:00
Fran Lozano
01331ed311
Update CHANGELOG.md
2020-10-16 19:08:30 +02:00
Fran Lozano
f638a3d50c
Store relation name in manifest's node object
2020-10-16 18:38:22 +02:00
Gerda Shank
512c41dbaf
Remove injected_sql. Store non-ephemeral injected_sql in compiled_sql
2020-10-15 11:52:03 -04:00
Github Build Bot
f6bab4adcf
Release dbt v0.18.1
2020-10-13 21:31:54 +00:00
Jeremy Cohen
526ecee3da
Merge pull request #2832 from fishtown-analytics/fix/colorama-upper-044
...
Set colorama upper bound to <0.4.4
2020-10-13 17:20:05 -04:00
Jeremy Cohen
1bc9815d53
Set colorama upper bound to <0.4.4
2020-10-13 16:26:10 -04:00
Ran Ever-Hadani
78bd7c9465
Eliminate asterisk from raw_execute to try an fix integration error
2020-10-11 12:06:56 -07:00
Ran Ever-Hadani
d74df8692b
Eliminate pep8 errors
2020-10-11 11:37:51 -07:00
Ran Ever-Hadani
eda86412cc
Accommodate first round of comments
2020-10-11 11:03:53 -07:00
Ran Ever-Hadani
cce5945fd2
Make partition metadata available to BigQuery users (rebased to dev/kiyoshi-kuromiya)
2020-10-10 17:44:07 -07:00
Drew Banin
72038258ed
Merge pull request #2805 from fishtown-analytics/feature/bigquery-oauth-token
...
Support BigQuery OAuth using a refresh token and client secrets
2020-10-09 14:55:00 -04:00
Drew Banin
056d8fa9ad
Merge branch 'dev/kiyoshi-kuromiya' into feature/bigquery-oauth-token
2020-10-09 14:07:27 -04:00
Gerda Shank
3888e0066f
Merge pull request #2813 from fishtown-analytics/feature/2510-save-args-run_results
...
Save args in run_results.json
2020-10-09 13:51:14 -04:00
Drew Banin
ee6571d050
Merge branch 'dev/kiyoshi-kuromiya' into feature/bigquery-oauth-token
2020-10-09 10:15:23 -04:00
Gerda Shank
9472288304
Save args in run_results.json
2020-10-08 17:39:03 -04:00
Jeremy Cohen
fd5e10cfdf
Merge pull request #2817 from zmac12/feature/addDebugQueryMethod
...
Feature/add debug query method
2020-10-08 10:30:31 -04:00
Jeremy Cohen
aeae18ec37
Merge pull request #2821 from joelluijmes/feature/hard-delete-revival
...
Re-instate hard-deleted records during snapshot
2020-10-08 10:27:42 -04:00
Zach McQuiston
03d3943e99
fixing linter problem in connections.py
2020-10-08 07:47:16 -06:00
Zach McQuiston
214d137672
adding entry to changelog
2020-10-08 07:45:35 -06:00
Joël Luijmes
83db275ddf
Added changelog entry
2020-10-08 15:26:51 +02:00
Joël Luijmes
b8f16d081a
Remove redundant 'dbt_valid_to is null' checks
2020-10-08 15:24:04 +02:00
Joël Luijmes
675b01ed48
Re-snapshot records that were invalidated through hard-delete
2020-10-08 12:45:35 +02:00
Joël Luijmes
b20224a096
Refactor test for hard-delete snapshots
2020-10-08 12:45:35 +02:00
Zach McQuiston
fd6edfccc4
Removing accidental whitespace addition
2020-10-07 19:30:48 -06:00
Zach McQuiston
4c58438e8a
removing errant reference to debug_query method
2020-10-07 19:29:49 -06:00
Zach McQuiston
5ff383a025
Adding type hint for debug_query
2020-10-07 19:25:00 -06:00
Zach McQuiston
dcb6854683
adding debug_query to base/impl.py enabling plugin authors to write their own debug_query
2020-10-07 19:20:16 -06:00
Drew Banin
e4644bfe5a
support providing a token directly; update method name
2020-10-07 15:34:57 -04:00
Jeremy Cohen
93168fef87
Merge pull request #2809 from mescanne/bigquery_invocation_id
...
Add invocation_id to BigQuery jobs
2020-10-07 10:41:39 -04:00
Jeremy Cohen
9832822bdf
Merge branch 'dev/kiyoshi-kuromiya' into bigquery_invocation_id
2020-10-07 09:45:21 -04:00
Mark Scannell
5d91aa3bcd
Updated feature request resolution.
2020-10-07 10:55:35 +01:00
Zach McQuiston
354ab5229b
adding new debug_query function to base debug task
2020-10-03 16:20:52 -06:00
Zach McQuiston
00de0cd4b5
adding new debug_query function to base adapter
2020-10-03 16:19:48 -06:00
Mark Scannell
26210216da
Only set invocation_id if tracking is enabled
2020-10-03 15:32:53 +01:00
Mark Scannell
e29c14a22b
updated changelog
2020-10-03 13:09:04 +01:00
Mark Scannell
a6990c8fb8
fixe up
2020-10-03 13:04:56 +01:00
Mark Scannell
3e40e71b96
Added dbt_invocation_id to BigQuery jobs
2020-10-03 13:01:25 +01:00
Gerda Shank
3f45abe331
Merge pull request #2799 from fishtown-analytics/feature/2765-save-manifest
...
write manifest when writing run_results
2020-10-01 16:12:15 -04:00
Gerda Shank
6777c62789
Merge pull request #2804 from fishtown-analytics/rpc-test-timeouts
...
Increase rpc test timouts to avoid local test failures
2020-10-01 16:10:59 -04:00
Github Build Bot
1aac869738
Merge remote-tracking branch 'origin/releases/0.18.1rc1' into dev/0.18.1
2020-10-01 16:52:51 +00:00
Github Build Bot
493554ea30
Release dbt v0.18.1rc1
2020-10-01 16:39:50 +00:00
Drew Banin
1cf87c639b
( #2344 ) Support BigQuery OAuth using a refesh token and client secrets
2020-09-30 23:04:40 -04:00
Gerda Shank
2cb3d92163
Increase rpc test timouts to avoid local test failures
2020-09-30 17:34:23 -04:00
Jeremy Cohen
89b6e52a73
Merge pull request #2791 from kingfink/tf/fix-snapshot-compilation-error
...
Fix snapshot compilation error
2020-09-30 16:54:45 -04:00
Tim Finkel
97407c10ff
revert dockerfile
2020-09-30 15:54:50 -04:00
Tim Finkel
81222dadbc
Merge branch 'tf/fix-snapshot-compilation-error' of https://github.com/kingfink/dbt into tf/fix-snapshot-compilation-error
2020-09-30 15:53:40 -04:00
Tim Finkel
400555c391
update tests
2020-09-30 15:52:12 -04:00
Tim Finkel
9125b05809
Update CHANGELOG.md
2020-09-30 14:46:45 -04:00
Jeremy Cohen
139b353a28
Merge pull request #2796 from Foxtel-DnA/feature/6434-bq-retry-rate-limit
...
UPDATE _is_retryable() to handle BQ rateLimitExceeded
2020-09-30 13:09:11 -04:00
Jared Champion (SYD)
fc474a07d0
REBASED on dev/0.18.1; moved CHANGELOG entries
2020-09-30 10:57:03 +10:00
championj-foxtel
8fd8fa09a5
Merge pull request #1 from fishtown-analytics/dev/0.18.1
...
Dev/0.18.1
2020-09-30 09:56:27 +10:00
Gerda Shank
41ae831d0e
write manifest when writing run_results
2020-09-29 16:35:44 -04:00
Gerda Shank
dbca540d70
Merge pull request #2781 from fishtown-analytics/feature/2700-improve-yaml-selector-errors
...
Add better error messages for yaml selectors
2020-09-29 16:16:44 -04:00
Tim Finkel
dc7eca4bf9
update changelog
2020-09-25 17:07:05 -04:00
Tim Finkel
fb07149cb7
fix snapshot compliation error
2020-09-25 16:52:14 -04:00
Github Build Bot
b2bd5a5548
Merge remote-tracking branch 'origin/releases/0.18.1b3' into dev/0.18.1
2020-09-25 20:20:21 +00:00
Github Build Bot
aa6b333e79
Release dbt v0.18.1b3
2020-09-25 20:05:31 +00:00
Jeremy Cohen
0cb9740535
Merge pull request #2789 from fishtown-analytics/fix/require-keyring
...
Fix: require keyring on snowflake
2020-09-25 15:00:12 -04:00
Gerda Shank
46eadd54e5
Add better error messages for yaml selectors
2020-09-25 14:53:30 -04:00
Jeremy Cohen
6b032b49fe
Merge branch 'dev/0.18.1' into fix/require-keyring
2020-09-25 14:13:53 -04:00
Jeremy Cohen
35f78ee0f9
Merge pull request #2754 from aiguofer/include_external_tables_in_get_columns_in_relation
...
Include external tables in get_columns_in_relation redshift adapter
2020-09-25 13:25:06 -04:00
Jeremy Cohen
5ec36df7f0
Merge branch 'dev/0.18.1' into include_external_tables_in_get_columns_in_relation
2020-09-25 12:52:39 -04:00
Jeremy Cohen
f918fd65b6
Merge pull request #2766 from jweibel22/fix/redshift-iam-concurrency-issue
...
Give each redshift client their own boto session
2020-09-25 12:50:20 -04:00
Jeremy Cohen
d08a39483d
PR feedback
2020-09-25 12:11:49 -04:00
Jeremy Cohen
9191f4ff2d
Merge branch 'dev/0.18.1' into fix/redshift-iam-concurrency-issue
2020-09-25 12:10:33 -04:00
Jeremy Cohen
19232f554f
Merge pull request #2785 from fishtown-analytics/feature/metadata-env-vars
...
add env vars with a magic prefix to the metadata
2020-09-24 17:03:30 -04:00
Jeremy Cohen
b4a83414ac
Require optional dep (keyring) on snowflake
2020-09-24 15:49:35 -04:00
Jeremy Cohen
cb0e62576d
Merge pull request #2732 from Mr-Nobody99/feature/add-snowflake-last-modified
...
Added last_altered query to Snowflake catalog macro
2020-09-24 15:29:04 -04:00
Alexander Kutz
e3f557406f
Updated test/integration/029_docs_generate_test.py to reflect new stat
2020-09-23 11:08:08 -05:00
Jacob Beck
676af831c0
add env vars with a magic prefix to the metadata
2020-09-23 10:04:06 -06:00
Jacob Beck
873d76d72c
Merge pull request #2786 from fishtown-analytics/feature/invocation-id
...
add invocation_id to artifact metadata
2020-09-23 10:03:42 -06:00
Jacob Beck
8ee490b881
Merge pull request #2749 from joelluijmes/snapshot-hard-deletes-joell
...
Include hard-deletes when making snapshot
2020-09-23 08:31:48 -06:00
Jacob Beck
ff31b277f6
add invocation_id to artifact metadata
2020-09-23 08:07:20 -06:00
Jacob Beck
120eb5b502
Merge pull request #2778 from fishtown-analytics/feature/common-artifact-metadata
...
Feature: common artifact metadata
2020-09-23 08:06:45 -06:00
Alexander Kutz
a93e288d6a
Added missing commaabove addition.
2020-09-22 17:48:48 -05:00
Alexander Kutz
8cf9311ced
Changed 'BASE_TABLE' to 'BASE TABLE'
2020-09-22 16:04:30 -05:00
Alexander Kutz
713e781473
Reset branch against dev/0.8.1 and re-added changes.
...
udpated changelog.md
2020-09-22 14:54:56 -05:00
Jacob Beck
a32295e74a
fix schema collection script
2020-09-22 13:48:31 -06:00
Jacob Beck
204b02de3e
fix freshness RPC response behavior
2020-09-22 12:49:17 -06:00
Jacob Beck
8379edce99
Add a common metadata field to JSON artifacts
...
Adjusted how schema versions are set
RPC calls no longer have the schema version in their replies
2020-09-22 10:35:02 -06:00
Github Build Bot
e265ab67c7
Merge remote-tracking branch 'origin/releases/0.18.1b2' into dev/0.18.1
2020-09-22 14:23:45 +00:00
Github Build Bot
fde1f13b4e
Release dbt v0.18.1b2
2020-09-22 14:09:51 +00:00
Jeremy Cohen
9c3839c7e2
Merge pull request #2782 from fishtown-analytics/docs/0.18.1-exposures
...
[revised] dbt-docs changes for v0.18.1
2020-09-22 09:51:04 -04:00
Jeremy Cohen
c0fd702cc7
Rename reports --> exposures
2020-09-22 08:44:58 -04:00
Jacob Beck
429419c4af
Merge pull request #2780 from fishtown-analytics/feature/rename-results-to-exposures
...
reports -> exposures
2020-09-21 15:15:11 -06:00
Jacob Beck
56ae20602d
reports -> exposures
2020-09-21 14:46:48 -06:00
Jacob Beck
a4b80cc2e4
Merge branch 'dev/kiyoshi-kuromiya' into snapshot-hard-deletes-joell
2020-09-21 14:11:01 -06:00
Jacob Beck
4994cc07a0
Merge pull request #2767 from fishtown-analytics/feature/schema-versions
...
Feature/schema versions
2020-09-21 14:10:28 -06:00
Joël Luijmes
e96cf02561
Merge remote-tracking branch 'upstream/dev/kiyoshi-kuromiya' into snapshot-hard-deletes-joell
2020-09-21 21:43:49 +02:00
Jacob Beck
764c9b2986
PR feedback
2020-09-21 11:56:07 -06:00
jweibel22
40c6499d3a
Update CHANGELOG.md
...
Co-authored-by: Jacob Beck <beckjake@users.noreply.github.com >
2020-09-20 13:31:43 +02:00
Jimmy Rasmussen
3a78efd83c
Add test cases to ensure default boto session is not used
2020-09-20 13:31:15 +02:00
Jimmy Rasmussen
eb33cf75e3
Add entry to CHANGELOG
2020-09-18 10:44:00 +02:00
Jimmy Rasmussen
863d8e6405
Give each redshift client their own boto session
...
since the boto session is not thread-safe, using the default session in a multi-threaded scenario will result in concurrency errors
2020-09-18 10:29:26 +02:00
Jacob Beck
1fc5a45b9e
Merge pull request #2768 from fishtown-analytics/fix/makefile-on-macos
...
fix the new makefile on macos
2020-09-17 13:42:32 -06:00
Github Build Bot
7751fece35
Merge remote-tracking branch 'origin/releases/0.18.1b1' into dev/0.18.1
2020-09-17 19:09:23 +00:00
Github Build Bot
7670c42462
Release dbt v0.18.1b1
2020-09-17 18:54:44 +00:00
Jacob Beck
b72fc3cd25
fix the new makefile on macos
2020-09-17 11:47:06 -06:00
Jacob Beck
4cc1a4f74c
changelog
2020-09-17 10:04:28 -06:00
Jacob Beck
540607086c
fix docker message
2020-09-17 10:03:32 -06:00
Jacob Beck
7d929e98af
Embed schema/dbt versions into the json schema for artifacts
2020-09-17 10:03:32 -06:00
Joël Luijmes
0086097639
Fix non-deterministic behavior by sorting on id (redshift test failed)
2020-09-17 11:55:23 +02:00
Jacob Beck
daff0badc8
Merge branch 'dev/0.18.1' into dev/kiyoshi-kuromiya
2020-09-16 14:16:33 -06:00
Jacob Beck
22c4d8fabe
Merge pull request #2741 from heisencoder/fix/docker-testing-on-linux
...
Fix docker-based testing for Linux users
2020-09-16 13:39:02 -06:00
Jeremy Cohen
3485482460
Merge pull request #2760 from fishtown-analytics/docs/0.18.1-reports
...
dbt-docs changes for v0.18.1
2020-09-16 15:21:54 -04:00
Jacob Beck
c43873379c
Merge pull request #2758 from fishtown-analytics/fix/version-bump
...
Bump version: 0.18.0 → 0.19.0a1
2020-09-16 13:17:58 -06:00
Jeremy Cohen
ea5e5df5a3
Support reports in dbt-docs
2020-09-16 13:44:17 -04:00
Jacob Beck
f2caf2f1ff
Merge pull request #2752 from fishtown-analytics/feature/reports
...
Feature: reports
2020-09-16 10:41:43 -06:00
Jacob Beck
07d4020fca
Bump version: 0.18.0 → 0.19.0a1
2020-09-16 10:34:53 -06:00
Jacob Beck
2142e529ff
PR feedback: make report selector more like source selector, remove reports from fqn slector
...
Make some corresponding fqn adjustments
Add dbt ls report output
Fix dbt ls source output
The default selector now also returns reports
Update tests
2020-09-16 07:23:36 -06:00
Joël Luijmes
b9d502e2e6
Ensure dbt_valid_to is latest column
2020-09-16 09:11:42 +02:00
Joël Luijmes
8c80862c10
Snapshot hard-delete tests also for bigquery, snowflake and redshift
2020-09-16 08:41:45 +02:00
Joël Luijmes
2356c7b63d
Use dict.get for optional paramater invalidate_hard_deletes
2020-09-16 07:51:01 +02:00
Diego Fernandez
9c24fc25f5
Add entry to CHANGELOG
2020-09-15 15:11:05 -06:00
Diego Fernandez
4f1a6d56c1
Include external tables in get_columns_in_relation redshift adapter
2020-09-15 15:09:55 -06:00
Joël Luijmes
b71b7e209e
Update changelog
2020-09-15 22:01:02 +02:00
Joël Luijmes
2581e98aff
Snapshot hard-delete opt-in during config
2020-09-15 16:47:16 +02:00
Joël Luijmes
afc7136bae
Fix rpc integration snapshot tests
2020-09-15 16:47:11 +02:00
Joël Luijmes
e489170558
Add test for snapshotting hard deleted records
2020-09-15 16:45:36 +02:00
Joël Luijmes
50106f2bd3
Include hard-deletes when making snapshot
...
It sets dbt_valid_to to the current snapshot time.
2020-09-15 16:45:36 +02:00
Jacob Beck
e96f4a5be6
got redshifted
2020-09-14 14:35:00 -06:00
Jacob Beck
4768ac5fda
Fix and add new tests, update changelog
2020-09-14 14:35:00 -06:00
Jacob Beck
c91fcc527a
add comparison/selector logic for reports
2020-09-14 12:24:00 -06:00
Jacob Beck
8520ff35b3
Add reports feature
...
Add ParsedReport/UnparsedReport
add report parser and report node logic to manifest/results/dbt ls/selectors
NonSourceNode -> ManifestNode
add GraphMemberNode type that includes reports
2020-09-14 11:36:31 -06:00
Jacob Beck
9b8a98f4ec
Test quality of life/cleanup
...
remove unused test folders
move rpc tests from 048 to 100 for convenience
- Migrating these to the test/rpc model is going to take work. In the
interim, developers can now use `tests/integeration/0*` to run all non-rpc
tests.
2020-09-14 10:26:02 -06:00
Jacob Beck
4bd4afaec7
bumpversion for 0.18.1
2020-09-11 14:21:03 -06:00
Matt Ball
69352d8414
Fix docker-based testing for Linux users
...
See https://github.com/fishtown-analytics/dbt/issues/2739
This change enables Linux users to run the dbt tests via the docker
image. It follows the recommendations from this article:
https://jtreminio.com/blog/running-docker-containers-as-current-host-user/
Notable changes:
* Added new Makefile rule to generate a .env file that contains USER_ID
and GROUP_ID environment variables to the ID of the current user. This
is in turn used by docker-compose and the Dockerfile to make the Docker
image run as the current user. (Note that on Windows or Mac, this
behavior happens by default).
* Reordered Dockerfile to allow for better caching of intermediate
images (i.e., put things that don't depend on ARGS earlier).
* Bumped CircleCI's Dockerfile from version 7 to 9. Jake rebuilt
9 off of this PR.
2020-09-11 11:15:18 -06:00
Jeremy Cohen
4a21ea6575
Merge pull request #2723 from tpilewicz/fix/freshness-logs
...
Feat(result logs): Use three logging levels
2020-09-09 15:03:06 -04:00
Thomas Pilewicz
86bbb9fe38
Add contributors section to 0.18.1
2020-09-09 18:50:39 +02:00
Thomas Pilewicz
4030d4fc20
Move logging levels changelog entry to 0.18.1
2020-09-09 18:50:27 +02:00
tpilewicz
182f69a9ec
Merge pull request #2 from fishtown-analytics/dev/0.18.1
2020-09-09 18:43:45 +02:00
Gerda Shank
fb40efe4b7
Merge pull request #2733 from fishtown-analytics/fix/2539-comment-quoting
...
When column config says quote, use quotes in SQL to add comments
2020-09-08 14:40:27 -04:00
Jacob Beck
9d00c00072
Merge pull request #2735 from fishtown-analytics/feature/include-unrendered-configs-2
...
Feature: include unrendered configs
2020-09-08 10:57:29 -06:00
Jacob Beck
10c3118f9c
Merge branch 'dev/kiyoshi-kuromiya' into feature/include-unrendered-configs-2
2020-09-08 09:29:09 -06:00
Jacob Beck
1fa149dca2
Merge pull request #2736 from fishtown-analytics/feature/rpc-state-defer
...
Feature: state and defer in RPC calls
2020-09-08 09:28:02 -06:00
Jacob Beck
60f4c963b5
Merge branch 'dev/kiyoshi-kuromiya' into feature/rpc-state-defer
2020-09-08 08:41:21 -06:00
Gerda Shank
51b8e64972
When column config says quote, use quotes in SQL to add comments
...
Add separate test for column comments. Fix Snowflake catalog comments.
2020-09-04 16:44:19 -04:00
Jacob Beck
ae542dce74
Merge dev/marian-anderson
2020-09-04 07:12:04 -06:00
Github Build Bot
fa8a4f2020
Merge remote-tracking branch 'origin/releases/0.18.0' into dev/marian-anderson
2020-09-03 16:45:49 +00:00
Github Build Bot
481bdd56d3
Release dbt v0.18.0
2020-09-03 16:02:36 +00:00
Github Build Bot
1a9083ddb7
Merge remote-tracking branch 'origin/releases/0.18.0rc2' into dev/marian-anderson
2020-09-03 15:52:16 +00:00
Github Build Bot
9779f43620
Release dbt v0.18.0rc2
2020-09-03 15:49:09 +00:00
Jacob Beck
d31e82edfc
this does not belong here
2020-09-02 10:13:41 -06:00
Jeremy Cohen
981535a1c3
Merge pull request #2734 from fishtown-analytics/docs/0.18.0-followup
...
dbt-docs changes for v0.18.0 (final)
2020-09-01 17:14:16 -04:00
Jacob Beck
5354e39e5f
add defer/state args to RPC, add tests
2020-09-01 14:25:28 -06:00
Jacob Beck
ca9293cbfb
changelog
2020-09-01 14:22:58 -06:00
Jeremy Cohen
e2fe6a8249
Add project-level overviews
2020-09-01 14:58:15 -04:00
Jeremy Cohen
a8347b7ada
Add missing changelog entry
2020-09-01 14:57:47 -04:00
Jacob Beck
bcbf7c3b7b
remove default values from unrendered configs
2020-09-01 10:30:03 -06:00
Jacob Beck
6a26cb280f
fix the tests, add unrendered configs for sources
2020-09-01 10:30:03 -06:00
Jacob Beck
fd658ace9d
Attach unrendered configs to parsed nodes
2020-09-01 10:30:03 -06:00
Jacob Beck
5e71a2aa3f
Add unrendered configs to project
2020-09-01 10:30:03 -06:00
Jacob Beck
e3fb923b34
removed v1 config
2020-09-01 10:30:03 -06:00
Jeremy Cohen
f55b257609
Merge pull request #2722 from genos/genos/fix-for-2347
...
fix for 2347
2020-09-01 08:50:04 -04:00
genos
81bf3dae5c
add contributors to changelog
2020-08-31 23:48:49 -04:00
Graham
d0074f3411
Merge branch 'dev/marian-anderson' into genos/fix-for-2347
2020-08-31 09:30:52 -04:00
Gerda Shank
2cc2d971c6
Merge pull request #2727 from fishtown-analytics/fix/2197-long-table-names
...
Check Postgres relation name lengths and throw error when over 63
2020-08-28 09:31:23 -04:00
Gerda Shank
5830f5590e
Tweak error message, reformat for flake8
2020-08-27 16:35:35 -04:00
Jacob Beck
75facebe80
Merge pull request #2726 from fishtown-analytics/fix/require-version-validation
...
Validate require-dbt-version before validating dbt_project.ymls chema
2020-08-27 10:01:13 -06:00
Jacob Beck
0130398e9f
Update core/dbt/config/project.py
...
Co-authored-by: Kyle Wigley <kyle@fishtownanalytics.com >
2020-08-27 08:16:33 -06:00
Gerda Shank
22d9b86e9f
update changelog for #2197
2020-08-26 14:15:01 -04:00
Gerda Shank
c87b671275
Use csv for data in test 063; tweak several lines
2020-08-26 13:57:03 -04:00
Jacob Beck
1eb5857811
add missing unit tests
2020-08-26 08:40:53 -06:00
Gerda Shank
5fc1cb39a6
Check Postgres relation name lengths and throw error when over 63
2020-08-26 10:28:37 -04:00
Jacob Beck
1f8e29276e
move the require-dbt-version check to before parsing
...
update changelog
2020-08-25 13:17:00 -06:00
Thomas Pilewicz
cf02c7fd02
Fix(get_printable_result): return type hints
2020-08-25 16:52:21 +02:00
Thomas Pilewicz
5d93c64c0e
Add logging levels to features of 0.18.0
2020-08-24 15:59:31 +02:00
Thomas Pilewicz
c738928ea3
Feat(result logs): Use three logging levels
2020-08-24 14:04:05 +02:00
genos
707310db64
update changelog
2020-08-23 17:40:36 -04:00
genos
59bf43dc1f
Fix for #2347
...
**Introduction**
This PR attempts to fix #2347 , wherein we wish `dbt` to complain about trying to install with a Python version < 3.6.
**Changes**
Per [the issue's suggestion](https://github.com/fishtown-analytics/dbt/issues/2347 ), I found every `setup.py` file I could:
```
-# If you have the fantastic `fd` utility installed:
fd setup.py
-# This also works
find . -name setup.py -print
```
Then to each of these, I added the following after the `import sys`:
```
if sys.version_info < (3, 6):
print('Error: dbt does not support this version of Python.')
print('Please upgrade to Python 3.6 or higher.')
sys.exit(1)
```
**Testing**
I used the [`nix` package manager](https://nixos.org ) to attempt installing this branch with both Python 2.7 and Python 3.8.
_Python 2.7_ fails as expected:
```
~/github/test2 ∃ cat default.nix
let
pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/20.03.tar.gz ") { };
py = pkgs.python27Full.withPackages (p: [ p.setuptools ]);
in pkgs.mkShell {
name = "python-2-env";
buildInputs = [ py ];
}
~/github/test2 ∃ nix-shell --pure
[nix-shell:~/github/test2]$ python ../dbt/setup.py build
Error: dbt does not support this version of Python.
Please upgrade to Python 3.6 or higher.
[nix-shell:~/github/test2]$ echo $?
1
```
_Python 3.8_ still works:
```
~/github/test3 ∃ cat default.nix
let
pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/20.03.tar.gz ") { };
py = pkgs.python38Full.withPackages (p: [ p.setuptools ]);
in pkgs.mkShell {
name = "python-3-env";
buildInputs = [ py ];
}
~/github/test3 ∃ nix-shell --pure
[nix-shell:~/github/test3]$ python ../dbt/setup.py build
running build
[nix-shell:~/github/test3]$ echo $?
0
```
2020-08-23 17:13:36 -04:00
Jacob Beck
fe461381a2
Merge pull request #2721 from fishtown-analytics/feature/more-test-only-adapter-methods
...
add more test helper methods
2020-08-21 13:13:28 -06:00
Jacob Beck
685873ab42
its ok to hit a deprecation with tracking disabled
2020-08-21 12:31:35 -06:00
Jacob Beck
b6a951903e
add more test helper methods
2020-08-21 08:02:45 -06:00
Github Build Bot
acfa84918e
Release dbt v0.18.0rc1
2020-08-19 20:10:33 +00:00
Jacob Beck
75304eb3be
Merge pull request #2718 from fishtown-analytics/fix/only-jinja-is-comments
...
When deciding if we should bypass rendering, also check for comments
2020-08-19 13:39:03 -06:00
Jacob Beck
1d7eb59ff2
Merge pull request #2712 from fishtown-analytics/feature/adapter-cte-generation
...
Feature: adapter cte generation
2020-08-19 12:40:04 -06:00
Jacob Beck
4273cc9e29
Merge pull request #2709 from kconvey/kconvey-copy-job
...
Add a BigQuery adapter macro to enable usage of CopyJobs
2020-08-19 12:39:49 -06:00
Jacob Beck
29be2de5cb
When deciding if we should bypass rendering, also check for comments
2020-08-19 12:27:40 -06:00
Jacob Beck
91b0496c89
Merge branch 'dev/marian-anderson' into kconvey-copy-job
2020-08-19 12:12:14 -06:00
Jacob Beck
f043f948de
Merge pull request #2711 from kconvey/kconvey-model-ttl
...
Support TTL for BigQuery tables
2020-08-19 12:10:14 -06:00
Jeremy Cohen
7ef7a8f306
Merge pull request #2708 from rsenseman/enhancement/color_output_command_line_flag_second_attempt
...
Add --use-colors cli option (second attempt)
2020-08-19 13:42:24 -04:00
Robert
db8eea2468
Merge branch 'dev/marian-anderson' into enhancement/color_output_command_line_flag_second_attempt
2020-08-19 10:01:12 -07:00
Kurt Convey
55813d9209
hours_to_expiration
2020-08-19 10:08:52 -06:00
Kurt Convey
4c05daae1b
Merge with origin
2020-08-19 09:41:37 -06:00
Kurt Convey
16eb7232c3
Check the copy model for failure
2020-08-19 09:18:33 -06:00
Kurt Convey
c28ffcdd9f
merge with origin
2020-08-19 09:08:07 -06:00
Kurt Convey
4b8652f1c4
Should be two results for original table and (failing) copy
2020-08-19 08:58:39 -06:00
Jeremy Cohen
674bd8f264
Merge pull request #2715 from fishtown-analytics/docs/0.18.0-search-selectors
...
Docs site updates for 0.18.0
2020-08-19 10:41:48 -04:00
Jeremy Cohen
1ba832dbfe
Docs site updates for 0.18.0
2020-08-18 18:42:32 -04:00
Jacob Beck
d3e4d3fbcb
pr feedback: remove commented out code
2020-08-18 14:34:12 -06:00
Jacob Beck
58a3cb4fbd
changelog update
2020-08-18 14:34:11 -06:00
Jacob Beck
8ad1551b15
when you think about it, data tests are really just ctes
2020-08-18 14:33:56 -06:00
Jacob Beck
123771163a
hide more things from the context
2020-08-18 14:33:56 -06:00
Jacob Beck
f80a759488
Have the adapter be responsible for producing the compiler
...
The adapter's Relation is consulted for adding the ephemeral model prefix
Also hide some things from Jinja
Have the adapter be responsible for producing the compiler, move CTE generation into the Relation object
2020-08-18 14:33:56 -06:00
Kurt Convey
42f8a4715e
Assert that single result has error
2020-08-18 14:11:28 -06:00
Jeremy Cohen
c29892e340
Merge pull request #2710 from fishtown-analytics/feature/add-deprecation-tracking
...
Track deprecation warnings
2020-08-18 16:11:02 -04:00
Kurt Convey
b4a2ed6bb5
Look for proper string
2020-08-18 14:06:53 -06:00
Kurt Convey
67e8caf045
No need to assert model success
2020-08-18 14:05:44 -06:00
Kurt Convey
2562debe31
Put --debug before run
2020-08-18 13:42:58 -06:00
Kurt Convey
87e2fd610c
Raise error for bad materializations and set a default
2020-08-18 13:41:00 -06:00
Kurt Convey
af118bcc53
Check stdout with --debug for actual ddl
2020-08-18 13:23:28 -06:00
Kurt Convey
7e01172b4c
Set status
2020-08-18 13:08:58 -06:00
Kurt Convey
f56ae93772
Return from copy_bq_table and fix test
2020-08-18 10:46:04 -06:00
Kurt Convey
3834805929
Use injected sql from results
2020-08-18 10:25:11 -06:00
Kurt Convey
50b6057bbf
Make copy a proper materialization
2020-08-18 10:17:15 -06:00
Kurt Convey
c0199abacf
Split failing and succeeding models
2020-08-18 08:34:41 -06:00
Kurt Convey
77688c74f3
Remove test from wrong PR
2020-08-17 17:20:24 -06:00
Kurt Convey
47ab7419ac
Embed profile name
2020-08-17 17:08:38 -06:00
Kurt Convey
dc209f77ec
Fix class name
2020-08-17 16:35:29 -06:00
Kurt Convey
76aa8c7df5
Fix class name
2020-08-17 16:32:22 -06:00
Kurt Convey
671a29ff34
plugins
2020-08-17 16:16:46 -06:00
Kurt Convey
eb35794aca
Fix table options string
2020-08-17 16:05:44 -06:00
Kurt Convey
a8d6691dee
Mock config better
2020-08-17 15:59:04 -06:00
Kurt Convey
955f4ae977
Add entry to CHANGELOG
2020-08-17 15:26:33 -06:00
Kurt Convey
25a869a686
Fix unit test
2020-08-17 15:23:25 -06:00
Kurt Convey
7aa8030b76
Fix test name
2020-08-17 15:12:02 -06:00
Kurt Convey
108d843bba
Add newlines
2020-08-17 15:10:51 -06:00
Kurt Convey
099fea8565
Add integration test
2020-08-17 15:04:59 -06:00
Jeremy Cohen
a573a2ada1
Explicitly import dbt.tracking module
2020-08-17 17:00:07 -04:00
Kurt Convey
1468ca8ebc
Expect to fail
2020-08-17 14:54:11 -06:00
Jeremy Cohen
274aea9f8f
Track deprecation warnings
2020-08-17 16:36:04 -04:00
Kurt Convey
38a99a75ed
Use correct models
2020-08-17 14:29:49 -06:00
Kurt Convey
6e06bd0cb4
Add unit test
2020-08-17 14:29:02 -06:00
Kurt Convey
02a793998a
Fix macro compilation
2020-08-17 14:10:04 -06:00
Kurt Convey
81ab9469b7
Add time_to_expiration
2020-08-17 14:00:10 -06:00
Kurt Convey
bcc928495d
Fix if statement
2020-08-17 13:13:48 -06:00
Kurt Convey
621ae7dbc9
make flake8 happy
2020-08-17 13:03:33 -06:00
Kurt Convey
85f2c03903
Tweak integration test
2020-08-17 12:26:25 -06:00
Kurt Convey
f7fd741d43
Attempt at integration tests
2020-08-17 12:16:20 -06:00
Kurt Convey
fad0d81837
Reference consts through the right module
2020-08-17 11:56:14 -06:00
Kurt Convey
09687409dc
move bq context to impl.py
2020-08-17 11:44:51 -06:00
Kurt Convey
091bcd107c
Fix test write dispositions
2020-08-17 09:20:22 -06:00
Kurt Convey
f9b300d63a
Add changelog
2020-08-17 09:12:52 -06:00
Jacob Beck
4f2acc2c96
Merge pull request #2703 from vogt4nick/2702-patch-redshift-table-size-estimation
...
patch redshift table size estimation
2020-08-17 09:04:03 -06:00
Kurt Convey
6aa4a60d5c
Add copy_job macro
2020-08-17 09:01:17 -06:00
rsenseman
b075bf51b0
one more changelog update
2020-08-16 14:48:08 -07:00
rsenseman
de2341ece0
update changelog
2020-08-16 14:44:55 -07:00
rsenseman
ff67e7d47c
add missing code; finalize pr
2020-08-16 14:34:46 -07:00
rsenseman
31644ed39d
initial commit
2020-08-16 13:48:55 -07:00
Jacob Beck
acb235ef4f
Merge pull request #2698 from fishtown-analytics/fix/snowflake-connector-python-upgrade
...
bump requirements, enable the token cache
2020-08-14 13:47:32 -06:00
Jacob Beck
d554835b50
Merge pull request #2695 from fishtown-analytics/feature/state-modified-selector
...
Add state:modified and state:new selectors
2020-08-14 13:44:48 -06:00
Jacob Beck
c8453d80fc
fix ls tests
2020-08-14 11:36:19 -06:00
Jacob Beck
f3f713ae65
fix macro change check to account for new list return value when macros are added/removed
2020-08-14 09:05:55 -06:00
Nick Vogt
cfa741e597
calculate Redshift table size in bytes, not megabytes
2020-08-14 11:02:10 -04:00
Jacob Beck
13fb2351ed
PR feedback
2020-08-14 08:48:59 -06:00
Jacob Beck
3555ba518d
fix unit tests
2020-08-12 14:07:20 -06:00
Jacob Beck
c5a19ca42e
bump requirements, enable the token cache
2020-08-12 14:00:37 -06:00
Jacob Beck
153eb7e9d3
flake8 update found more things to complain about
2020-08-12 07:44:52 -06:00
Jacob Beck
7ba52d4931
mypy is rightfully mad about our funky inheritance, just copy+paste things
...
We can fix this when we drop python 3.6 and unions stop collapsing types
2020-08-12 07:44:52 -06:00
Jacob Beck
ebe5b46653
Add state:modified and state:new selectors
2020-08-12 07:44:51 -06:00
Jacob Beck
1bd82d4914
Merge pull request #2694 from kconvey/kconvey-retry-upstream
...
Add retry of additional errors
2020-08-12 07:43:50 -06:00
Jeremy Cohen
89775fa94f
Merge pull request #2594 from brunomurino/issue-2533
...
added option --adapter to dbt init, to create sample profiles.yml bas…
2020-08-11 18:27:02 -04:00
Kurt Convey
4456872635
Didn't forget to add myself to the contributors
2020-08-11 15:23:02 -06:00
Kurt Convey
ee9ae22651
Fix variable name
2020-08-11 15:16:15 -06:00
Kurt Convey
afe0f46768
use isinstance with tuple
2020-08-11 15:15:04 -06:00
Kurt Convey
203d8c3481
Update CHANGELOG
2020-08-11 12:42:31 -06:00
Kurt Convey
c9ae49255d
Add test from dev
2020-08-11 12:29:52 -06:00
Bruno Murino
51f17d3358
Update plugins/redshift/dbt/include/redshift/sample_profiles.yml
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2020-08-10 13:38:08 +01:00
Bruno Murino
cacdd58b41
Update plugins/redshift/dbt/include/redshift/sample_profiles.yml
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2020-08-10 13:38:01 +01:00
Bruno Murino
8b7bcbbc47
Update plugins/postgres/dbt/include/postgres/sample_profiles.yml
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2020-08-10 13:37:49 +01:00
Bruno Murino
bdf9482e75
Update plugins/postgres/dbt/include/postgres/sample_profiles.yml
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2020-08-10 13:37:42 +01:00
Bruno Murino
b94d0b66e6
Apply suggestions from code review
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2020-08-06 19:11:12 +01:00
Bruno Murino
1dff122a94
Update plugins/postgres/dbt/include/postgres/sample_profiles.yml
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2020-08-06 19:10:34 +01:00
Jacob Beck
fb8065df27
Merge pull request #2686 from fishtown-analytics/feature/override-core-macros
...
include project macros in the manifest the adapter stores locally
2020-08-06 11:35:25 -06:00
Jacob Beck
4274139210
Fix the test
...
Make sure "dbt deps" reloads the full manifest
Make sure commands that reload the dbt_project.yml properly reset the config (including adapters)
2020-08-05 14:50:59 -06:00
brunomurino
b422a44c03
update redshift and snowflake sample profiles
2020-08-05 19:56:29 +01:00
Jacob Beck
285479c0bc
include project macros in the manifest the adapter stores locally
2020-08-05 09:35:00 -06:00
Jacob Beck
e641ec12fa
Merge pull request #2684 from fishtown-analytics/feature/dispatch-schema-tests
...
convert tests
2020-08-05 09:27:55 -06:00
Jeremy Cohen
36fda28a92
Merge pull request #2677 from bbhoss/bq_impersonate
...
Add support for impersonating a service account with BigQuery
2020-08-04 14:37:53 -04:00
Jacob Beck
1ece515074
Merge pull request #2679 from fishtown-analytics/feature/adapter-dispatch
...
Create adapter.dispatch
2020-08-04 12:01:00 -06:00
Jacob Beck
04f840d907
convert tests
2020-08-04 09:47:48 -06:00
Preston Marshall
df8ccc04eb
add contributors section
2020-08-04 10:49:35 -04:00
Jacob Beck
dd764b93e0
check if packages is a string and error out
2020-08-04 07:46:50 -06:00
Jacob Beck
335497f688
fixed error message, fixed the changelog
2020-08-03 15:48:04 -06:00
Jacob Beck
41a9251982
pr feedback: better errors for dotted macro names
...
add more tests
2020-08-03 12:31:29 -06:00
Preston Marshall
84bf03580d
Merge remote-tracking branch 'upstream/dev/marian-anderson' into bq_impersonate
2020-08-03 13:00:13 -04:00
Preston Marshall
43d5dfcb71
move changelog additions to next
2020-08-03 12:57:30 -04:00
Preston Marshall
7a9fc7ef12
update changelog
2020-08-03 12:55:42 -04:00
Jacob Beck
ecf24cd4d9
adapter_macro -> adapter.dispatch
...
Added tests
Added deprecation warning + tests
2020-08-03 10:49:28 -06:00
Jacob Beck
ba828e59de
Merge pull request #2675 from fishtown-analytics/feature/python-adapter-macro
...
Port the adapter macro to python
2020-08-03 09:50:13 -06:00
Jacob Beck
34719f94b4
PR feedback
2020-08-03 09:15:28 -06:00
Preston Marshall
c95a6792e5
add a test
2020-08-01 15:40:48 -04:00
Preston Marshall
88529d5c25
first try
2020-08-01 15:12:00 -04:00
Jacob Beck
3489878395
changelog
2020-07-31 12:19:51 -06:00
Jacob Beck
d938013733
remove unused to_dict method
2020-07-31 12:19:51 -06:00
Jacob Beck
dfe2ea4d0a
adapter macro -> python
...
Updated tests
2020-07-31 12:19:51 -06:00
Jacob Beck
7a3e345cf7
upgrade mypy, now it might not crash on failures
2020-07-31 12:19:51 -06:00
Jacob Beck
1fb8c9c896
Merge pull request #2656 from fishtown-analytics/feature/defer-to-prod
...
Feature/defer to prod
2020-07-31 12:09:25 -06:00
Jacob Beck
2ed97640d5
changelog updates
2020-07-31 09:24:51 -06:00
Jacob Beck
6d31aa6f67
PR feedback
...
- Actually set deferred=True
- add a test for that
- combine/unify environment vars vs cli vars behavior between defer/state better
- make the state path start with DBT_
2020-07-31 09:24:31 -06:00
Jacob Beck
a76ffbd194
add logging, add tests, update changelog
2020-07-31 09:24:31 -06:00
Jacob Beck
0d7e0ba087
read json and merge things
2020-07-31 09:24:31 -06:00
Github Build Bot
618d491f92
Release dbt v0.18.0b2
2020-07-30 17:11:32 +00:00
Jacob Beck
21a3462798
Merge pull request #2589 from azhard/dbt-2586-bigquery-policy-tags
...
Added support for setting policy tags for columns in BigQuery
2020-07-30 11:07:16 -06:00
Azhar Dewji
d2c5783e5a
Addressed PR commments
2020-07-30 11:02:23 -04:00
Azhar Dewji
007cb4516a
Addressed PR comments
2020-07-30 10:55:52 -04:00
Azhar Dewji
8ec93d9973
bigquery integration test fix
2020-07-30 10:55:52 -04:00
Azhar Dewji
0a64708302
Linting fix
2020-07-30 10:55:52 -04:00
Azhar Dewji
b964f6dd22
Rebased
2020-07-30 10:55:52 -04:00
Azhar Dewji
14233188be
Changed update_column to update_columns
2020-07-30 10:55:51 -04:00
Azhar Dewji
e3193c6243
Updated pull request #
2020-07-30 10:55:51 -04:00
Azhar Dewji
8782714ec3
Added support for setting policy tags for columns in BigQuery
2020-07-30 10:55:51 -04:00
Jacob Beck
03010ecde7
Merge branch 'dev/0.17.2' into dev/marian-anderson
2020-07-29 13:54:10 -06:00
Github Build Bot
97d63d3da4
Release dbt v0.17.2
2020-07-29 19:09:25 +00:00
Jacob Beck
a212f4eb5e
Merge pull request #2663 from fishtown-analytics/fix/changelog
...
Fix the changelog
2020-07-29 12:47:26 -06:00
Jacob Beck
5c993fd568
fix the changelog
2020-07-29 12:10:35 -06:00
Jeremy Cohen
df60189266
Merge pull request #2658 from fishtown-analytics/docs/0.17.2-code-block-background
...
Docs site updates for 0.17.2: fix code block background
2020-07-29 10:50:36 -04:00
Jacob Beck
97f61a9ecc
Merge pull request #2654 from fishtown-analytics/fix/redshift-schema-override-catalog
...
fix redshift catalog with custom schemas
2020-07-29 08:39:16 -06:00
Jeremy Cohen
8740e6ca66
Docs site updates for 0.17.2: fix code block background
2020-07-28 18:43:09 -04:00
Claire Carroll
c9eb3c2b54
Merge pull request #2623 from fishtown-analytics/asset-paths
...
Compile assets as part of docs generate
2020-07-28 17:15:21 -04:00
Github Build Bot
d9411bc9bb
Release dbt v0.17.2rc1
2020-07-28 20:20:16 +00:00
Claire Carroll
a3a1b143cf
Compile assets as part of docs generate
2020-07-28 15:49:15 -04:00
Jacob Beck
5c18c78b50
Merge pull request #2650 from fishtown-analytics/fix/close-connections
...
Fix: close connections after use
2020-07-28 13:25:24 -06:00
Jacob Beck
3bcb5f147d
fix a test with too-strict bounds, any <=2 value is ok here
2020-07-28 13:02:34 -06:00
Jacob Beck
1bfe43ff76
PR feedback:
...
- fix conftest adainivalue_line calls
- deprecate the release argument to execute_macro
2020-07-28 12:07:08 -06:00
Jacob Beck
2d2019c059
add a test for cross-schema catalog generation, fix redshift to work for schema overrides
2020-07-28 11:08:24 -06:00
Jacob Beck
4613619841
missed the snowflake rpc tests
2020-07-28 07:55:24 -06:00
Jacob Beck
44e3c7eb6d
only try to cancel open connections
2020-07-27 16:08:43 -06:00
Jacob Beck
fd0b460391
alter the tests so we can run rpc tests on snowflake
2020-07-27 15:16:14 -06:00
Jacob Beck
3b917b9d79
Handle the fallout of closing connections in release
...
- close() implies rollback, so do not call it
- make sure to not open new connections for executors in single-threaded mode
- logging cleanups
- fix a test case that never acquired connections
- to cancel other connections, one must first acquire a connection for the master thread
- change a number of release() calls to rollback
release vs rollback
2020-07-27 15:16:14 -06:00
Jacob Beck
970049fde8
Always close connections in release()
2020-07-27 15:16:13 -06:00
Jacob Beck
5b40cc4a01
Merge pull request #2649 from fishtown-analytics/fix/dbt-clean-noprofile
...
Fix: dbt clean without profile
2020-07-27 15:15:13 -06:00
Jacob Beck
0919c5642d
Update CHANGELOG.md
...
Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com >
2020-07-27 13:42:21 -06:00
Jacob Beck
de4f90b3f4
clean without a profile
2020-07-27 11:11:38 -06:00
Jacob Beck
a1c2270956
azure pipelines silently messing with $PATH again
2020-07-27 11:04:50 -06:00
Jacob Beck
b9cf0cfc44
Merge pull request #2646 from joshpeng-quibi/fix/fail-fast-output
...
Fix fast-fail not logging original error message
2020-07-27 11:02:26 -06:00
joshpeng-quibi
d2869e4749
Apply suggestions from code review
...
Co-authored-by: Jacob Beck <beckjake@users.noreply.github.com >
2020-07-24 13:49:18 -07:00
Jacob Beck
3ec911b62c
Merge pull request #2640 from fishtown-analytics/feature/yaml-selections
...
Feature/yaml selections
2020-07-24 14:30:32 -06:00
Josh Peng
4ff66c1cb6
Fix fast-fail not logging original error message
2020-07-24 12:06:57 -07:00
Jacob Beck
184146b825
add changelog
2020-07-22 09:57:27 -06:00
Jacob Beck
1e8543efb1
Create selector config file, handle selector argument
...
Added tests
Added RPC and CLI support
2020-07-22 09:31:55 -06:00
Jacob Beck
a976e54c19
Parse selectors
...
This doesn't use hologram, as it needs more permissive schemas than what hologram provides.
Added some unit tests
2020-07-22 07:58:10 -06:00
Jacob Beck
88e26da026
Merge pull request #2629 from fishtown-analytics/feature/new-node-selectors
...
new node selectors (#2425 )
2020-07-21 20:33:48 -06:00
Jacob Beck
58522b8f67
PR feedback: catch non-declared config items
2020-07-21 15:43:52 -06:00
Github Build Bot
f09bb17da4
Release dbt v0.17.2b1
2020-07-21 13:28:43 +00:00
Jacob Beck
353ae0f090
changelog fix
2020-07-21 07:25:42 -06:00
Jacob Beck
0404881d85
Merge pull request #2635 from fishtown-analytics/feature/debug-logging-env-vars
...
add environment variables for logging
2020-07-20 13:48:18 -06:00
Jacob Beck
fb88d54c31
Merge pull request #2628 from fishtown-analytics/feature/advanced-node-selection
...
split up node selection
2020-07-20 13:48:04 -06:00
Jacob Beck
a523356504
new node selectors
...
- added a bunch of unit tests
- added some new integration tests
- cleaned up some selection logic
2020-07-20 11:16:40 -06:00
Jacob Beck
91ff7f1c80
PR feedback: only log about the innermost failed expectation
2020-07-20 11:16:17 -06:00
Jacob Beck
51f6c268d3
add environment variables for logging
2020-07-20 09:55:19 -06:00
Jacob Beck
5e1b284ad0
Bump version: 0.17.1 → 0.17.2a1
2020-07-20 09:39:04 -06:00
Github Build Bot
a962d28674
Release dbt v0.17.1
2020-07-20 12:48:01 +00:00
Jacob Beck
7f0c1f823a
Merge pull request #2627 from kning/ISSUE-2569_bad_target
...
Add more helpful error message for misconfiguration in profiles.yml
2020-07-17 11:48:08 -06:00
Kenny Ning
d08a2b8c02
Add PR link to changelog
2020-07-17 11:49:38 -04:00
Kenny Ning
d7f3518a9a
Add test
2020-07-17 11:37:02 -04:00
Jacob Beck
d34d2a023c
pr feedback: Make UniqueId a NewType
2020-07-17 09:03:39 -06:00
Kenny Ning
1ecd954199
Address review
2020-07-16 16:08:02 -04:00
Jacob Beck
6e161ab84e
azure pipelines silently messing with $PATH again
2020-07-15 17:25:14 -06:00
Jacob Beck
97c59f36a2
fix some bugs
...
rpc, ls, some import tweaks
2020-07-15 16:30:38 -06:00
Jacob Beck
89443eac0e
Refactor for node selection
...
- moved what remains of the linker into compilation, migrate things to the graph
- compile uncompiled ephemeral models when prepending ctes
- no more including them all in the selection
- per-node task selectors
- added the idea of a selection spec, graph selection uses that
- right now we parse include + exclude strings
2020-07-15 16:30:38 -06:00
Jacob Beck
3e8414e259
merge dev/0.17.1
2020-07-15 15:49:40 -06:00
Kenny Ning
06a30f0447
Add more helpful error message for misconfiguration in profiles.yml
2020-07-15 17:26:12 -04:00
Jacob Beck
4e636aa53a
Merge pull request #2610 from fishtown-analytics/feature/adapter-test-suite-changes
...
move the sql for getting different rows into dbt proper
2020-07-15 08:43:59 -06:00
Claire Carroll
7bd18ef331
Merge pull request #2576 from fishtown-analytics/update-logo
...
Update logo
2020-07-10 10:24:52 -04:00
Github Build Bot
181fcc3965
Release dbt v0.17.1rc4
2020-07-08 16:12:55 +00:00
Jacob Beck
3a1f745e65
Merge pull request #2618 from fishtown-analytics/fix/native-env-opt-in
...
make native env rendering opt-in
2020-07-08 09:44:57 -06:00
Jacob Beck
2dc3ea3f12
fix tests
2020-07-08 08:41:58 -06:00
Jacob Beck
1e8ed49661
Apply suggestions from code review
...
Co-authored-by: Drew Banin <drew@fishtownanalytics.com >
2020-07-08 08:34:39 -06:00
Jacob Beck
84bf169458
make native env rendering opt-in
2020-07-07 14:50:40 -06:00
brunomurino
3dabe62254
updated redshift and snowflake sample_profiles.yml
2020-07-04 14:13:51 +01:00
brunomurino
0d246ac95b
updated postgres sample_profiles.yml
2020-07-02 21:16:45 +01:00
brunomurino
2d0612c972
updated bigquery sample_profiles.yml
2020-07-02 21:14:52 +01:00
Jacob Beck
4cf2b78fca
move the sql for getting different rows into dbt proper, from the test suite. Bump pytest dependency.
2020-07-02 08:39:23 -06:00
Jacob Beck
3af8a22d13
Merge pull request #2590 from fishtown-analytics/fix/adapter-macro-namespacing
...
Fix adapter macro namespacing (#2548 )
2020-07-01 10:51:29 -06:00
Jacob Beck
b0c7b3a38b
remove commented out code
2020-07-01 09:21:01 -06:00
Github Build Bot
3be03b681b
Release dbt v0.17.1rc3
2020-07-01 13:19:07 +00:00
Jacob Beck
dd5b7240b6
Merge pull request #2601 from fishtown-analytics/fix/bigquery-seed-descriptions
...
On bigquery, persist docs for seeds as well as tables/views
2020-06-30 12:26:52 -06:00
Jacob Beck
2b9b82e5c4
Merge branch 'dev/0.17.1' into fix/bigquery-seed-descriptions
2020-06-30 09:38:13 -06:00
Jacob Beck
c5c14bb798
Merge pull request #2603 from fishtown-analytics/fix/windows-is-the-worst-os-ever
...
remove length check + catch any exceptions on windows
2020-06-30 09:04:26 -06:00
Jacob Beck
a201fc773a
Merge pull request #2599 from fishtown-analytics/fix/more-jinja-quoting-sadness
...
Fix more jinja quoting sadness
2020-06-30 07:53:42 -06:00
Jacob Beck
d897164d5c
PR feedback: add exception to the log message
2020-06-30 07:53:23 -06:00
Jacob Beck
e558d3f9ad
remove length check + catch any exceptions on windows
2020-06-30 07:42:41 -06:00
Jacob Beck
6a7078f634
flake8
2020-06-29 16:02:49 -06:00
Jacob Beck
b14676f0c5
Merge branch 'dev/0.17.1' into fix/more-jinja-quoting-sadness
2020-06-29 15:57:07 -06:00
Jacob Beck
4cef1af8cb
On bigquery, persist docs for seeds as well as tables/views
2020-06-29 15:50:48 -06:00
Jacob Beck
42aa407adc
PR feedback: extra test
2020-06-29 15:00:00 -06:00
Claire Carroll
e7298d60d3
Merge pull request #2600 from fishtown-analytics/fix/package-tracking
...
Hash local package name
2020-06-29 16:38:32 -04:00
Claire Carroll
9c0ff932c8
Update changelog
2020-06-29 15:56:03 -04:00
Claire Carroll
288dd221f8
Hash local package name
2020-06-29 15:39:17 -04:00
Jacob Beck
d374ef9ffe
If it comes into literal_eval as a string and it comes out as a string, return the original
2020-06-29 13:18:52 -06:00
Jeremy Cohen
7fa6a363b2
Merge pull request #2577 from fishtown-analytics/refactor/contributing
...
Revise CONTRIBUTING doc
2020-06-26 14:42:47 -04:00
brunomurino
13da3390e5
updated changelog
2020-06-26 18:58:45 +01:00
brunomurino
4164e6ee8e
updated setup.py of adapters to include sample_profiles.yml
2020-06-26 18:12:18 +01:00
brunomurino
2b454f99dd
updated 040_init_test for profile postgres to work with new --adapter option of dbt init
2020-06-25 23:54:33 +01:00
brunomurino
2025634417
added option --adapter to dbt init, to create sample profiles.yml based on chosen adapter
2020-06-25 23:27:08 +01:00
Github Build Bot
42e582a6ad
Release dbt v0.17.1rc2
2020-06-25 21:05:41 +00:00
Drew Banin
339a7e9409
Merge pull request #2588 from fishtown-analytics/docs/new-logo-jun-2020
...
Docs/new logo jun 2020
2020-06-25 17:02:52 -04:00
Jacob Beck
f420911e19
Merge pull request #2591 from fishtown-analytics/fix/windows-path-length-nonsense
...
More windows path lengths
2020-06-25 14:57:09 -06:00
Jacob Beck
2e39a5fff0
handle unmarked path too long errors
2020-06-25 10:32:42 -06:00
Jacob Beck
e0f7deabfd
spelling
2020-06-25 10:26:09 -06:00
Jacob Beck
abe345e925
Allow plugin macros to override core
...
Moved some relation logic out of adapters and into contracts
Fixed a typo in a type name
Fixed an issue where tests could fail based on color settings
Fixed the type analysis for adapter plugins/factory
Swapped more concrete tyeps out for protocols
Finally removed PACKAGES global
Added unit tests
2020-06-25 10:05:41 -06:00
Jacob Beck
4d33554465
protocols
2020-06-25 10:02:03 -06:00
Jacob Beck
32c559838d
Try to make imports a little more sane, ordering-wise
...
consolidate dbt.ui, move non-rpc node_runners into their tasks
move parse_cli_vars into config.utils
get rid of logger/exceptions requirements in dbt.utils
2020-06-25 10:02:02 -06:00
Jacob Beck
62a0bf8732
store package info on the factory instead of globally
...
adapter_types -> plugins
Use factory.packages instead of global PACKAGES
2020-06-25 09:59:47 -06:00
Jacob Beck
24ae8b4498
Merge pull request #2555 from DrMcTaco/snowflake-query-tag
...
Snowflake query tag
2020-06-25 08:46:27 -06:00
Jacob Beck
2945a91189
Merge branch 'dev/marian-anderson' into snowflake-query-tag
2020-06-25 07:36:45 -06:00
Drew Banin
26735dac8d
update docs logo
2020-06-25 09:35:13 -04:00
Jeremy Cohen
7c0a41c998
Fixup anchor
2020-06-25 00:35:39 -04:00
Jeremy Cohen
778a89ef8a
Feedback from Andrew
2020-06-25 00:34:31 -04:00
Jeremy Cohen
83d85dc928
Merge pull request #2581 from brunomurino/issue-2437
...
added option to specify profile when connecting to redshift via IAM
2020-06-24 10:13:34 -04:00
Dan Rubin
25a06aee01
Merge branch 'dev/marian-anderson' into snowflake-query-tag
2020-06-24 09:16:07 -04:00
Dan Rubin
c89ea1ad56
fix CHANGELOG formatting
2020-06-24 09:15:17 -04:00
Jeremy Cohen
76f9f23b16
Merge pull request #2530 from alepuccetti/bigquery-processed-bytes-log
...
Add data processed info into dbt run logs for all statement types
2020-06-23 16:39:33 -04:00
Jacob Beck
a45f6bea5b
Merge pull request #2584 from fishtown-analytics/fix/plus-prefixed-hooks
...
Fix prefixing support to not render "+pre-hook"
2020-06-23 14:37:08 -06:00
Dan Rubin
49fec95b33
Merge branch 'dev/marian-anderson' into snowflake-query-tag
2020-06-23 16:20:44 -04:00
Jacob Beck
d38fd80dff
PR feedback: double seeds
2020-06-23 13:25:01 -06:00
Jacob Beck
37b03ef5d2
Fix prefixing support to not render "+pre-hook"
...
Increased the startup timeout in the RPC tests to try to fix some local timeouts
2020-06-23 10:04:23 -06:00
Bruno Murino
0c878a056e
Update CHANGELOG.md
...
Co-authored-by: Jacob Beck <beckjake@users.noreply.github.com >
2020-06-23 16:24:25 +01:00
Alessandro Puccetti
1da4bef068
Update CHANGELOG.md
...
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com >
2020-06-23 16:47:37 +02:00
brunomurino
65c156750d
updated changelog
2020-06-23 15:09:10 +01:00
brunomurino
bd34dfe1f7
removed hardcoded option in fetch_cluster_credentials
2020-06-23 14:53:53 +01:00
brunomurino
78445f9879
updates to make pep8 compliant
2020-06-22 23:15:40 +01:00
brunomurino
053210330e
updated to comply with pep8 guidelines
2020-06-22 23:03:14 +01:00
brunomurino
df894f91f2
updated changelog
2020-06-22 22:27:57 +01:00
Jacob Beck
f758614b0a
Merge pull request #2579 from christineberger/feature/change_yml_warning_color
...
Feature: Change project-level warning color
2020-06-22 14:41:39 -06:00
Bruno Murino
71846c98b3
added option to specify profile when connecting to redshift via IAM
2020-06-22 21:39:03 +01:00
Alessandro Puccetti
272cad7b97
core/dbt/utils: remove extra empty line
2020-06-22 22:21:02 +02:00
Alessandro Puccetti
5eb9523325
plugins/bigquery: log processed bytes value for all statement types
...
Fixes https://github.com/fishtown-analytics/dbt/issues/2526
2020-06-22 22:21:02 +02:00
Alessandro Puccetti
d2b2f2f6b6
core/dbt/utils: add format_rows_number
2020-06-22 22:21:01 +02:00
Alessandro Puccetti
55f9720a44
core/dbt/utils: add PB to format_bytes and always return the accurate value
2020-06-22 22:17:06 +02:00
Jeremy Cohen
9d52e6acbf
Feedback from Sanjana
2020-06-22 14:57:05 -04:00
Jacob Beck
8f649f5ab4
Merge pull request #2547 from azhard/bigquery-alter-column-type
...
Add support for altering BigQuery column types
2020-06-22 12:50:18 -06:00
Christine
fbfb54e3e2
Fixed flake8 suggestions on inline comment
2020-06-22 12:51:39 -05:00
Christine
e523d908bf
Moved dbt.tracking InvocationProcesser in to print_run_end_messages() to avoid import loop
2020-06-22 12:45:24 -05:00
Christine
8b2d416035
Simplified warning_tag() logic
2020-06-22 11:56:51 -05:00
Jeremy Cohen
078356458b
Feedback from Drew
2020-06-22 12:38:01 -04:00
Christine
4846246f12
Fixed indenting on end cap of function
2020-06-22 11:30:33 -05:00
Christine
dc10b3d904
Fixed more flake8 whitespace suggestions
2020-06-22 11:27:59 -05:00
Christine
05d4a5e4a3
Updated old warning_tag concatenation to use new warning_tag function
2020-06-22 11:17:46 -05:00
Christine
2883b6dc17
Removed some formatting and whitespace
2020-06-22 11:16:31 -05:00
Christine
5f9e153db8
Fixed whitespace suggestions from flake8
2020-06-22 11:04:09 -05:00
Christine
c05153f00a
Added changes to changelog.md
2020-06-22 09:38:05 -05:00
Claire Carroll
3616641e93
Update other logos
2020-06-22 08:48:59 -04:00
Christine
78cbeca606
Added some whitespace under the deprecation warning
2020-06-20 15:09:06 -05:00
Christine
354c7ecddf
Changed configuration paths warning to be wrapped with new warning_tag
2020-06-20 15:08:44 -05:00
Christine
4d659318c4
Changed format of project-level warning messages to be wrapped with new warning_tag
2020-06-20 15:07:58 -05:00
Christine
9d4399eb28
Added a warning_tag wrapper fuction to printer class
2020-06-20 15:06:48 -05:00
Github Build Bot
32702559f6
Release dbt v0.17.1rc1
2020-06-19 21:13:16 +00:00
Drew Banin
310a1dfaec
Merge pull request #2565 from fishtown-analytics/docs/0.17.1-docs-updates
...
bump docs site
2020-06-19 17:08:55 -04:00
Drew Banin
6528b5f25b
Merge branch 'dev/0.17.1' into docs/0.17.1-docs-updates
2020-06-19 17:08:44 -04:00
Jacob Beck
19378abfae
Merge pull request #2575 from fishtown-analytics/fix/backport-aliasing
...
backport aliases in config fix
2020-06-19 15:06:57 -06:00
Jacob Beck
593e86c15d
Merge pull request #2550 from bodschut/bodschut/bq_nested_column_descriptions
...
Add functionality to persist descriptions to nested columns in bigquery
2020-06-19 13:58:21 -06:00
Jeremy Cohen
6226d60c9a
Give CONTRIBUTING.md some love
2020-06-19 15:36:38 -04:00
Claire Carroll
aa35dc9437
Update logo
2020-06-19 15:25:28 -04:00
Jacob Beck
098d05c27f
Fix tests
2020-06-19 12:54:10 -06:00
Jacob Beck
05917c6d2f
update changelog
2020-06-19 11:29:35 -06:00
Jacob Beck
e300c62d76
add a test
2020-06-19 11:24:47 -06:00
Jacob Beck
3829d1f964
Fix a regression where dbt ignored aliases in config() calls
2020-06-19 11:16:07 -06:00
Bob De Schutter
0d2593e0e3
Fix docs_generate test
2020-06-19 18:19:50 +02:00
Bob De Schutter
e99f0d12b8
Remove commented out code
...
Co-authored-by: Jacob Beck <beckjake@users.noreply.github.com >
2020-06-19 16:56:17 +02:00
Jacob Beck
d0e736f31b
Merge branch 'dev/0.17.1' into bodschut/bq_nested_column_descriptions
2020-06-19 08:41:19 -06:00
Bob De Schutter
3a5db3c16d
Updated changelog
2020-06-19 16:37:53 +02:00
Jacob Beck
0cddd9c417
Merge pull request #2566 from fishtown-analytics/fix/windows-long-paths
...
Fix windows long paths
2020-06-19 08:31:18 -06:00
Bob De Schutter
51ce3e1093
Fix persist docs functionality for bigquery to include descriptions for nested columns
2020-06-19 16:31:14 +02:00
Jacob Beck
db50880399
fix this test for spawn() vs fork()
2020-06-19 07:26:40 -06:00
Jacob Beck
8e63da2f04
Be pickier about when to mangle paths, handle the case where we still hit path length limits
2020-06-18 17:11:15 -06:00
Drew Banin
a0b502dd44
Merge branch 'dev/0.17.1' into docs/0.17.1-docs-updates
2020-06-18 17:34:42 -04:00
Jacob Beck
ff272c797a
make sure dbt clean still works...
2020-06-18 15:17:27 -06:00
Jacob Beck
f4b93c8a5a
handle long paths in windows, add a test
2020-06-18 15:08:43 -06:00
Jacob Beck
c01056b99a
Merge pull request #2559 from fishtown-analytics/fix/alias-bug
...
Fix a regression where dbt ignored aliases in config() calls
2020-06-18 07:13:36 -06:00
Jeremy Cohen
12ca721e0b
Merge pull request #2567 from fishtown-analytics/feature/ci-on-forks
...
Run all integration tests on PRs from forks
2020-06-17 18:57:07 -04:00
Drew Banin
9c7d519ab2
changelog
2020-06-17 16:32:57 -04:00
Drew Banin
5299079f01
bump docs site
2020-06-17 16:30:30 -04:00
Jeremy Cohen
4c8575dea3
Build PRs on forks
2020-06-17 15:01:50 -04:00
Dan Rubin
f3aaa8bcee
Merge branch 'snowflake-query-tag' of https://github.com/DrMcTaco/dbt into snowflake-query-tag
2020-06-17 13:02:18 -04:00
Dan Rubin
2cc53321ae
remove incorrect 2nd arg to config.get
2020-06-17 13:01:16 -04:00
Jacob Beck
64e9d704e2
Merge pull request #2554 from fishtown-analytics/fix/parallel-rpc-method-stomping
...
Fix parallel rpc methods (2484)
2020-06-17 10:54:09 -06:00
Jacob Beck
1c1f10056d
Fix a regression where dbt ignored aliases in config() calls
2020-06-17 07:41:19 -06:00
Dan Rubin
ccae4a8888
Merge branch 'dev/marian-anderson' into snowflake-query-tag
2020-06-16 18:49:08 -04:00
Dan Rubin
5c1840914b
update CHANGELOG.md
2020-06-16 18:43:58 -04:00
Dan Rubin
be61e6724a
add query tag update macros to each snowflake materialization
2020-06-16 18:43:44 -04:00
Dan Rubin
38975ecee0
add snowflake macros to support fetching current session query tag, updating the session query tag, and reverting to the old query tag
2020-06-16 18:43:26 -04:00
Dan Rubin
d8b31c02e2
add query tag to the snowflake adapter config
2020-06-16 18:43:13 -04:00
Dan Rubin
331b961819
add query tag to snoflake connection dataclass and set tag on new connection if query_tag value is present
2020-06-16 18:42:56 -04:00
Azhar Dewji
769e4d5193
Added warning comment to alter macro
2020-06-16 18:11:08 -04:00
Jacob Beck
dbf7e690b8
this test difference appears to be spawn vs fork
2020-06-16 16:00:07 -06:00
Jacob Beck
12a9c2b441
changelog
2020-06-16 14:46:07 -06:00
Jacob Beck
12d3c52de2
add a test
...
For the test to work, we have to switch to spawn so that we don't deadlock
2020-06-16 14:43:39 -06:00
Jacob Beck
614f0b4350
deepcopy args in method calls
2020-06-16 13:08:37 -06:00
Jacob Beck
74c78ef927
Merge branch 'dev/0.17.1' into dev/marian-anderson
2020-06-15 13:17:43 -06:00
Jacob Beck
765c2de148
Merge pull request #2537 from fishtown-analytics/fix/bump-minimum-python
...
Update minimum python version
2020-06-15 12:18:33 -06:00
Jacob Beck
644fef1e84
Update minimum python version: 3.6.2 has a typing bug that dbt/hologram handle poorly
2020-06-15 11:30:29 -06:00
Jacob Beck
b9053a24dc
Merge pull request #2535 from fishtown-analytics/feature/allow-package-version-floats
...
allow floats for package versions, add a test with a float-like branch (#2518 )
2020-06-15 11:29:55 -06:00
Jacob Beck
0be8326351
Merge pull request #2534 from fishtown-analytics/fix/dbt-modules-in-project-dir
...
Make deps respect --project-dir
2020-06-15 11:29:21 -06:00
Azhar Dewji
a7a73df696
Add support for altering BigQuery column types
2020-06-15 12:54:27 -04:00
Jacob Beck
e9b4c47887
PR feedback: use the project root instead of cwd
2020-06-15 10:07:58 -06:00
Jacob Beck
7ee6f1de39
allow relative paths in project-dir
2020-06-12 13:00:00 -06:00
Jacob Beck
eea51bee30
Merge pull request #2517 from azhard/bigquery-auth-views
...
Added support for BigQuery authorized views
2020-06-11 14:17:25 -06:00
Jacob Beck
0beb03fe53
allow floats for package versions, add a test with a float-like branch
2020-06-11 11:43:47 -06:00
Azhar Dewji
4ccfe020a2
Moved bigquery imports into bigquery test
2020-06-11 13:30:36 -04:00
Azhar Dewji
19735d5416
Added noqa flag to GrantTarget import
2020-06-11 13:19:05 -04:00
Azhar Dewji
e53a45b3e0
Added integration test
2020-06-11 13:10:10 -04:00
Jacob Beck
e00d1ed78c
make tests drop more things at cleanup
...
Make the unique ID a bit more collision-resistant to try to avoid collisions
2020-06-11 09:58:06 -06:00
Jacob Beck
83a4562b98
Make deps respect --project-dir
2020-06-11 07:39:12 -06:00
Jacob Beck
27fef50919
Merge pull request #2528 from fishtown-analytics/fix/do-not-create-schemas
...
Fix/do not create schemas
2020-06-11 07:36:15 -06:00
Jacob Beck
947a00f6c9
Bump version: 0.17.0 → 0.17.1a1
2020-06-10 15:11:31 -06:00
Jacob Beck
2f36d7d259
Update changelog
2020-06-10 15:08:46 -06:00
Jacob Beck
e2d7027074
add a test
2020-06-10 15:07:15 -06:00
Jacob Beck
a2de92d2a3
Fix azure pipelines builds by using the existing postgres setup
2020-06-10 15:07:15 -06:00
Jacob Beck
edef65f333
do not create schemas
2020-06-10 15:07:15 -06:00
Azhar Dewji
e7dfe97ecc
Debug message change
2020-06-10 14:33:29 -04:00
Azhar Dewji
cd0fe51109
Re-add dropped changelog lines
2020-06-10 11:55:51 -04:00
Azhar Dewji
6c0a5021cd
Merge branch 'dev/marian-anderson' into bigquery-auth-views
2020-06-10 11:54:38 -04:00
Jacob Beck
c9b34681b7
Merge pull request #2521 from azhard/bigquery-rename-relation
...
Added support for BigQuery relation renaming
2020-06-10 09:48:28 -06:00
Azhar Dewji
8422c3a9d3
grant_access no longer common and takes in grant_target_dict param
2020-06-10 11:35:18 -04:00
Azhar Dewji
daed251d0d
Fix seed file name
2020-06-09 17:04:42 -04:00
Azhar Dewji
ddb6d791e8
Fixed CI errors
2020-06-09 16:28:00 -04:00
Azhar Dewji
107170163a
Fixed relation creation
2020-06-09 15:00:29 -04:00
Azhar Dewji
7aa09e393c
Undo accidental commented out test
2020-06-09 14:50:08 -04:00
Azhar Dewji
75cc3e40a5
Addressed PR comments
2020-06-09 14:43:59 -04:00
Azhar Dewji
78cf84310c
Changed dataset_id to relation
2020-06-09 11:58:24 -04:00
Azhar Dewji
0f6622fab6
Added support for BigQuery relation renaming
2020-06-09 11:17:17 -04:00
Azhar Dewji
bb20df0db9
#1718 - Added support for BigQuery authorized views
2020-06-08 17:30:26 -04:00
Github Build Bot
8a2b5f53f6
Release dbt v0.18.0b1
2020-06-08 21:27:13 +00:00
Jacob Beck
e30db9816e
Merge branch 'dev/octavius-catto' into dev/marian-anderson
2020-06-08 14:52:29 -06:00
Github Build Bot
5f8efc3aa8
Release dbt v0.17.0
2020-06-08 17:12:47 +00:00
Jacob Beck
f3565f3f70
Merge pull request #2513 from southpolemonkey/dev/marian-anderson
...
(#2511 ) Remove log messages about opening connections in set_connecti…
2020-06-08 10:06:34 -06:00
Chenxuan Rong
2be7ee6ee0
update CHANGELOG
2020-06-08 23:42:22 +10:00
Chenxuan Rong
f7a8cd1250
( #2511 ) Move opening connection message to LazyHandle module
2020-06-08 15:17:07 +10:00
Chenxuan Rong
3f5cc224a6
( #2511 ) Move opening connection message inside open method
2020-06-05 08:18:36 +10:00
Jacob Beck
26363638bc
Merge pull request #2505 from aburgel/move-pytest-logbook
...
Move pytest-logbook dependency to dev_requirements.txt
2020-06-04 10:37:30 -06:00
Chenxuan Rong
1cd51181eb
( #2511 ) Remove log messages about opening connections in set_connection_name.
2020-06-05 00:53:02 +10:00
Alex Burgel
a1d5c340e8
Update changelog
2020-06-04 10:33:11 -04:00
Alex Burgel
6f1763317a
Update changelog
2020-06-04 10:31:24 -04:00
Alex Burgel
8973393333
Move pytest-logbook dependency to dev_requirements.txt
2020-06-04 10:31:24 -04:00
Jacob Beck
a5d53c4453
Add dbt:0.17.0rc4 dockerfiles and requirements
2020-06-02 08:19:47 -06:00
Jacob Beck
85fda89413
Bump version: 0.17.0rc3 → 0.17.0rc4
2020-06-02 07:48:09 -06:00
Jacob Beck
7b6c11adb7
Changelog update for 0.17.0rc4
2020-06-02 07:47:38 -06:00
Jacob Beck
7e14666802
Merge pull request #2502 from fishtown-analytics/fix/homebrew-packaging
...
Generate the homebrew formula pypi urls in the build script (#2500 )
2020-06-02 07:11:07 -06:00
Jacob Beck
6fbf6721b9
Merge pull request #2509 from fishtown-analytics/fix/snowflake-get-columns-missing-relation
...
fix get_columns_in_relation on snowflake (#2504 )
2020-06-02 06:34:52 -06:00
Jacob Beck
ec76498686
PR feedback: remove extra class def
2020-06-02 06:34:40 -06:00
Jacob Beck
05634e61a7
Merge pull request #2508 from scarrucciu/patch-1
...
Fix --dependency arg bug in create_adapter_plugin
2020-06-01 12:43:29 -06:00
Spencer Carrucciu
2dd7ef5302
Update CHANGELOG.md
2020-06-01 12:47:15 -04:00
Jacob Beck
49a5a556e9
fix get_columns_in_relation on snowflake
...
Also add a test for adapter methods to catch this if it regresses
2020-06-01 10:28:42 -06:00
Spencer Carrucciu
855dd98c62
Fix --dependency arg bug in create_adapter_plugin
...
Fix bug in create_adapter_plugin when a dependency is not passed in.
2020-06-01 07:37:33 -04:00
Jacob Beck
2759c9e3bd
Generate the homebrew formula pypi urls in the build script, check versions
2020-05-29 14:50:09 -06:00
Jacob Beck
594b8b786a
Merge pull request #2501 from fishtown-analytics/fix/better-filename-errors
...
add project names/filenames/values to rendering errors (#2499 )
2020-05-29 11:30:28 -06:00
Jacob Beck
3387d085b4
add project names/filenames/values to rendering errors
2020-05-29 10:21:38 -06:00
Jacob Beck
ee6b842f9a
Merge pull request #2491 from fishtown-analytics/fix/pin-protobufs
...
Put protobuf in version jail
2020-05-27 10:52:07 -06:00
Jacob Beck
fce11d7b46
Pin protobuf
2020-05-27 10:00:16 -06:00
Jacob Beck
48c9b8abdc
Merge branch 'fix/catalog-allow-none' into dev/octavius-catto
2020-05-27 08:01:05 -06:00
Jacob Beck
c3c99f317e
Merge pull request #2485 from Raalsky/direct-nodes-selector
...
Direct child model selector
2020-05-26 16:18:55 -06:00
Jacob Beck
9deb50f812
Add dbt:0.17.0rc3 dockerfiles and requirements
2020-05-26 14:55:16 -06:00
Jacob Beck
18c02ea991
Bump version: 0.17.0rc2 → 0.17.0rc3
2020-05-26 14:51:11 -06:00
Jacob Beck
3e23a5ff75
Merge pull request #2489 from fishtown-analytics/fix/catalog-allow-none
...
Make databases optional
2020-05-26 14:50:11 -06:00
Jacob Beck
076bb9c356
Allow column stats description fields to be empty
2020-05-26 14:06:31 -06:00
Jacob Beck
c21af17a30
Make databases optional
2020-05-26 11:59:20 -06:00
Raalsky
1d298ea5cf
Added type annotations and errors types refactored
2020-05-26 19:03:29 +02:00
Raalsky
f214ebf4e9
Graph methods refactor
2020-05-26 17:55:45 +02:00
Raalsky
ce5ca6f845
Updated CHANGELOG
2020-05-24 17:05:06 +02:00
Raalsky
cc2be00abb
Integration tests added for direct child model selection
2020-05-24 16:56:39 +02:00
Raalsky
bbfcce1cc6
Unit tests added for direct model selection
2020-05-24 16:42:53 +02:00
Raalsky
d81961ccee
flake8
2020-05-24 16:07:49 +02:00
Raalsky
c1da5dd513
Added direct child syntax to node selector
2020-05-24 14:08:22 +02:00
Raalsky
7c201fed81
Added internal methods from networkx library
2020-05-24 12:10:15 +02:00
Raalsky
f3a97b6c08
Initial refactor of ancestors/descendants selector methods
2020-05-24 12:03:53 +02:00
Jacob Beck
6509067856
Add dbt:0.17.0rc2 dockerfiles and requirements
2020-05-22 14:22:13 -06:00
Jacob Beck
6482718dbf
Bump version: 0.17.0rc1 → 0.17.0rc2
2020-05-22 14:18:45 -06:00
Jacob Beck
6dac4c76eb
Merge pull request #2481 from fishtown-analytics/fix/docs-perf
...
Improve docs performance (#2480 )
2020-05-22 14:16:43 -06:00
Jacob Beck
0d825eb590
Check for instances of "}}" and "%}" as well when deciding whether to render so users get compile errors instead of wrong data in that case
2020-05-22 13:34:32 -06:00
Jacob Beck
6cc0d9103d
Add changelog entry, push another entry to its real release + add contributors entry
2020-05-21 16:55:06 -06:00
Jacob Beck
514a1ad7d0
On windows, docs now include their original carriage returns before newlines
2020-05-21 16:53:17 -06:00
Jacob Beck
7f99353ca0
add libyaml-dev so we get that libyaml goodness
...
And point tests to the new image
2020-05-21 16:53:17 -06:00
Jacob Beck
a58657403e
avoid double-building the flat graph - we already built it at parse time
2020-05-21 16:53:17 -06:00
Jacob Beck
b9a86df795
if we have "cyaml" available, use it
...
This is a significant performance improvement with a lot of yaml files!
2020-05-21 16:53:17 -06:00
Jacob Beck
cf94ce6a54
add caching on the manifest for docs, ref, and source calls
...
removed the internal `find_*_by_name` calls for things that use the cache, just do them through `resolve_*`
fix unit tests to point at `resolve_*` now
2020-05-21 16:53:17 -06:00
Jacob Beck
e9d112ac39
Improve docs performance, especially on docs blocks that do not use jinja internally
...
Make docs parsing avoid a bunch of template nonsense and just use get_rendered
Add a check to get_rendered that skips the block if we can prove it should render to its input
- but not in native mode
2020-05-21 16:53:17 -06:00
Jacob Beck
f4c6bf30b6
Merge pull request #2358 from ChristianKohlberg/feature/snowflake-expose-query-id
...
Expose Snowflake query id in case of an exception raised by connector
2020-05-21 11:35:42 -06:00
Jacob Beck
626522d1bb
Merge pull request #2477 from fishtown-analytics/fix/global-vars
...
Fix some issues around global-scoped vars
2020-05-21 07:47:14 -06:00
Jacob Beck
58f3905af4
Merge pull request #2478 from fishtown-analytics/fix/source-patching-perf
...
fix source patching perf with no patches
2020-05-21 07:18:45 -06:00
ChristianKohlberg
89b4984009
Merge branch 'dev/octavius-catto' into feature/snowflake-expose-query-id
2020-05-20 23:15:38 +02:00
Christian Kohlberg
804c779359
expose snowflake query id if available
2020-05-20 23:12:05 +02:00
ChristianKohlberg
d699add216
Update CHANGELOG.md
...
Co-authored-by: Jacob Beck <beckjake@users.noreply.github.com >
2020-05-20 21:15:18 +02:00
Jacob Beck
e36fb3cc17
clean up replace() calls
2020-05-20 12:55:06 -06:00
Jacob Beck
fcf57a5066
if there is no patch, do not patch it
2020-05-20 12:48:38 -06:00
Jacob Beck
715c886dad
appease flake8/mypy
...
Change variables named `l` to `log`
Make IsFQNResource runtime type-checkable and use that to handle the FQN check
Make the GenerateNameProvider its own top-level Provider type
2020-05-20 11:34:14 -06:00
Jacob Beck
e2cd45b18a
Fix some issues around global-scoped vars
...
Fix global-level var rewriting from v2 -> v1
Add top-level vars to more lookup fields
When a user calls var() in a `generate_*_name` macro, on failure it now raises (like at runtime)
When a node has no FQN, pretend the FQN is just the package name when returning vars
Add tests
2020-05-20 10:59:15 -06:00
Jacob Beck
a8f8708418
Merge pull request #2475 from dmateusp/dmateusp/fix_docker_entrypoints
...
Fix ENTRYPOINT to use 'dbt'
2020-05-20 10:40:34 -06:00
Daniel Mateus Pires
ca5b02af9a
Add Contributors
2020-05-20 16:32:06 +01:00
Daniel Mateus Pires
832d43efdc
Add CHANGELOG note
2020-05-20 15:24:32 +01:00
Daniel Mateus Pires
e9daff7616
Fix ENTRYPOINT to use 'dbt'
2020-05-20 15:15:26 +01:00
Jacob Beck
22999a9861
Merge pull request #2467 from fishtown-analytics/fix/macos-tests
...
Fix tests to run on macos
2020-05-18 17:03:57 -06:00
Jacob Beck
64edf70b93
Fix tests to run on macos
2020-05-18 14:10:09 -06:00
Jacob Beck
75dbb0bc19
Merge pull request #2431 from alf-mindshift/dev/octavius-catto
...
Make it possible to hide macros in docs
2020-05-18 08:19:23 -06:00
Jacob Beck
e34439dcd7
Merge branch 'dev/marian-anderson' into dev/octavius-catto
2020-05-18 08:19:07 -06:00
Jacob Beck
9c84ce1e6f
Merge pull request #2461 from fishtown-analytics/fix/typo-failfast-msg
...
Fix typo in failfast msg
2020-05-18 08:11:04 -06:00
Jacob Beck
907003881e
Merge pull request #2464 from fishtown-analytics/fix/duplicate-sources-crash
...
Fix duplicate sources crash
2020-05-18 08:10:35 -06:00
Jacob Beck
4e2ec6b7f8
Merge pull request #2417 from Raalsky/feature/model-intersection-syntax
...
Model intersection syntax
2020-05-18 08:09:14 -06:00
Alf Lervag
4b90ff892f
Updated tests and changelog
2020-05-18 15:25:52 +02:00
Alf Lervag
5591a82a15
Make it possible to hide macros in docs
2020-05-18 15:21:25 +02:00
Raalsky
598c06f5b0
CHANGELOG update
2020-05-16 00:12:38 +02:00
Raalsky
bf12001c04
Added concatenation and intersection unittests
2020-05-16 00:06:52 +02:00
Raalsky
e59df0fbb5
Added tests for model intersection syntax
2020-05-16 00:06:52 +02:00
Raalsky
47f5c51b2d
Working prototype for model intersection syntax
2020-05-16 00:06:52 +02:00
Raalsky
e9b982b8ea
Initial tests for model intersection syntax
2020-05-16 00:06:51 +02:00
Jacob Beck
770cf71c13
Merge branch 'dev/octavius-catto' into dev/marian-anderson
2020-05-15 15:00:55 -06:00
Jacob Beck
5734fb897c
Merge pull request #2462 from fishtown-analytics/fix/version-2-test-intermittent-failures
...
Fix config-version 2 test intermittent failures
2020-05-15 15:00:33 -06:00
Jacob Beck
df8aa642c5
Fix crash on duplicate sources, add a test
2020-05-15 14:04:41 -06:00
Jacob Beck
5c76cfb071
remove the need for strict=False on most tests that have dependencies by upgrading the dependencies to config-version: 2
2020-05-15 11:58:26 -06:00
Jeremy Cohen
75f8bc2679
Fix typo in failfast msg
2020-05-15 13:36:51 -04:00
Jacob Beck
208c1cfb1a
reset deprecation warnings after tests, clean up the fallout
2020-05-15 09:36:00 -06:00
Jacob Beck
1d543b3737
Merge pull request #2455 from fishtown-analytics/fix/adapter-plugin-config-version
...
set config-version: 2 in the generated plugin project
2020-05-14 15:05:39 -06:00
Jacob Beck
b41b72c316
set config-version: 2 in the generated plugin project
2020-05-14 14:07:42 -06:00
Jacob Beck
26703df4d7
Merge pull request #2450 from fishtown-analytics/fix/status-line-relations
...
In dbt run, log relations for the status line instead of explicit node info
2020-05-14 12:24:16 -06:00
Jacob Beck
f490f6af98
In dbt run, log relations for the status line instead of explicit node parts
2020-05-14 11:09:16 -06:00
Claire Carroll
70ad551cf4
Merge pull request #2446 from fishtown-analytics/slack-references
...
Replace slack references
2020-05-14 06:36:16 -04:00
Jacob Beck
48705b6d8c
Merge pull request #2448 from fishtown-analytics/fix/bigquery-create-schema-macro
...
Fix create_schema macro on bigquery (#2445 )
2020-05-13 14:40:49 -06:00
Jacob Beck
6e5c8750f6
Fix create_schema macro on bigquery
2020-05-13 13:35:11 -06:00
Claire Carroll
1f78f30756
Replace slack references
2020-05-13 13:56:27 -04:00
Drew Banin
df5b97c614
Merge pull request #2427 from RealAshampoo/feature/bigquery-cost-limit
...
Add BigQuery option to define a maximum query cost limit
2020-05-13 13:05:21 -04:00
Jacob Beck
1de892bf0e
Merge pull request #2442 from fishtown-analytics/fix/empty-persist-docs
...
Backport #2440 to 0.17
2020-05-13 08:47:20 -06:00
Jacob Beck
457cbbd954
Merge pull request #2440 from fishtown-analytics/fix/empty-persist-docs
...
Fix empty SQL in persist docs (#2439 )
2020-05-13 08:39:32 -06:00
haukeduden
842edd62a6
Merge branch 'dev/octavius-catto' into feature/bigquery-cost-limit
2020-05-13 09:55:11 +02:00
Jacob Beck
7b374a4b08
Merge pull request #2438 from fishtown-analytics/feature/full-refresh-config
...
make full refresh a config item
2020-05-12 15:28:19 -06:00
Jacob Beck
d0f5664358
PR feedback - config is still not a dict
2020-05-12 14:49:29 -06:00
Jacob Beck
053546402d
Only run alter_column_comment if there are columns defined
...
Also remove unused (and confusing/misleading) test data
2020-05-12 14:42:37 -06:00
Jacob Beck
0fa9d0ab12
make full refresh a config item
...
Added integration tests, fixed existing tests
2020-05-12 12:10:03 -06:00
Jacob Beck
9d0eab6305
Bump version: 0.17.0rc1 → 0.18.0a1
2020-05-12 11:04:43 -06:00
Jacob Beck
79a90d0671
Add dbt:0.17.0rc1 dockerfiles and requirements
2020-05-12 10:51:35 -06:00
Jacob Beck
451e95e48d
Bump version: 0.17.0b1 → 0.17.0rc1
2020-05-12 10:45:59 -06:00
Jacob Beck
d3f535a0a7
Changelog update
2020-05-12 10:45:17 -06:00
Jacob Beck
4fc54dae32
Merge pull request #2435 from fishtown-analytics/fix/no-testing-homebrew
...
Do not test homebrew packages
2020-05-12 10:44:46 -06:00
Jacob Beck
4e70d078ef
Do not test homebrew packages
2020-05-12 09:50:37 -06:00
Drew Banin
31673d8acd
Merge pull request #2433 from fishtown-analytics/docs/0.17.0-hide-from-search
...
Docs site updates for 0.17.0: Hide hidden models from search
2020-05-12 11:03:31 -04:00
Drew Banin
d5e127bfea
Docs site updates for 0.17.0: Hide hidden models from search
2020-05-12 10:30:41 -04:00
Hauke Duden
f40374deb9
Merge branch 'dev/octavius-catto' into feature/bigquery-cost-limit
2020-05-12 16:15:22 +02:00
Hauke Duden
f8cbea4dbf
added whitespace around operator to pass flake8 tests
2020-05-12 15:37:19 +02:00
Hauke Duden
e3bd6cd5ef
- added pull request link to changelog
...
- added entry to contributors list
2020-05-12 15:35:57 +02:00
Drew Banin
71e7b0ab71
Merge pull request #2429 from fishtown-analytics/fix/track-package-deps-for-hub-site
...
Track project id for deps events
2020-05-12 09:32:51 -04:00
Drew Banin
fd662a1855
Merge pull request #2422 from azhard/fix/bigquery-view-spacing
...
Fix for extra spacing and parentheses for BigQuery views
2020-05-12 09:30:36 -04:00
Drew Banin
2a8940415d
quiet new pep8 check
2020-05-11 20:57:00 -04:00
Drew Banin
b63a271d55
( #23551 ) track project id for deps events
2020-05-11 20:48:28 -04:00
Hauke Duden
47e70efca6
updated changelog
2020-05-11 20:16:58 +02:00
Hauke Duden
0f3dd88029
- added maximum_bytes_billed option to bigquery profiles
...
- added basic unit test for maximum_bytes_billed
2020-05-11 20:06:25 +02:00
Azhar Dewji
35a052f4ab
Updated CHANGELOG links
2020-05-08 15:39:40 -04:00
Azhar Dewji
a31e78817e
Re-add dropped semi-colon
2020-05-08 15:38:45 -04:00
Azhar Dewji
f250d7ab47
Fix for extra spacing and parentheses for BigQuery views
2020-05-08 15:36:29 -04:00
Jacob Beck
8686ab9a9d
Merge pull request #2420 from fishtown-analytics/fix/dbt-init-v2-project
...
Fix dbt init to clone a v2 project (#2404 )
2020-05-08 09:18:08 -06:00
Jacob Beck
2cd98c2c60
Fix dbt init to clone a v2 project
2020-05-08 08:41:28 -06:00
Jacob Beck
c9eec4fa06
Merge pull request #2411 from fishtown-analytics/fix/always-preserve-case
...
Preserve the case of schemas and databases when listing relations (#2403 )
2020-05-08 08:15:07 -06:00
Drew Banin
6230c608d0
Merge pull request #2419 from fishtown-analytics/may-2020-update-readme
...
Update README.md
2020-05-07 23:11:12 -04:00
Jacob Beck
ab8392b856
Update core/dbt/task/runnable.py
...
Co-authored-by: Drew Banin <drew@fishtownanalytics.com >
2020-05-07 20:02:41 -06:00
Jacob Beck
067f02f9d8
Merge pull request #2418 from fishtown-analytics/fix/plugin-versions-search
...
fix version-loading logic to respect multiple plugins at the same path (#2410 )
2020-05-07 19:10:04 -06:00
Drew Banin
0c1f8550b8
Update README.md
2020-05-07 20:07:02 -04:00
Jacob Beck
7287204279
fix up the unit tests
2020-05-07 14:32:52 -06:00
Jacob Beck
e392212c0e
Change list_relations_without_caching macro to take a single argument
...
The argument is a Relation object with no identifier field, configured with the appropriate quoting information
Unique quoted/unquoted representations will be treated as distinct
The logic for generating what schemas to search for relations is now distinct from the catalog search logic.
Schema creation/dropping takes a similar relation argument
Add tests
2020-05-07 14:13:18 -06:00
Jacob Beck
dacfe38429
fix version-loading logic to respect multiple plugins at the same path
2020-05-07 14:08:43 -06:00
Jacob Beck
1eb75d9be7
Merge pull request #2416 from mikaelene/patch-1
...
Update CHANGELOG.md
2020-05-07 13:06:23 -06:00
Mikael Ene
a122d35e43
Update CHANGELOG.md
...
Added myself as contributor
2020-05-07 20:52:33 +02:00
Jacob Beck
445543e256
Merge pull request #2343 from Raalsky/warn-nothing-selected
...
Added warning to nodes selector if nothing was matched
2020-05-07 11:12:03 -06:00
Rafał Jankowski
cfeffe40f6
Typos corrected
2020-05-07 18:12:17 +02:00
Jacob Beck
dfb72ccb2e
Merge pull request #2413 from azhard/fix/bigquery-list-schemas
...
Fix error when trying to call list_schemas macro for BigQuery adapter
2020-05-07 09:56:56 -06:00
Jacob Beck
0f4a369aeb
Merge pull request #2414 from mikaelene/alias_returned_columns
...
alias for schema tests macros
2020-05-07 09:55:41 -06:00
Rafał Jankowski
385f7f2c24
Merge branch 'dev/octavius-catto' into warn-nothing-selected
2020-05-07 17:44:08 +02:00
Raalsky
aa36ed5aa4
CHANGELOG updated
2020-05-07 17:39:17 +02:00
Rafał Jankowski
c87096993e
Nodes selector refactor and added warning for non-existence of models for spec
2020-05-07 17:36:52 +02:00
Drew Banin
784ab79eec
Merge pull request #2402 from fishtown-analytics/feature/bigquery-column-comments
...
Feature/bigquery column comments
2020-05-07 10:55:09 -04:00
Drew Banin
7122a3f90b
Add column comments on BigQuery for views, tables, and incrementals
2020-05-07 09:48:18 -04:00
Mikael Ene
868a447c58
alias for schema tests macros
2020-05-07 07:38:21 +02:00
Azhar Dewji
002244e3b3
Fixed CHANGELOG links and added contributors section
2020-05-06 21:41:47 -04:00
Azhar Dewji
5a319d6ee1
Updated CHANGELOG.md
2020-05-06 17:21:51 -04:00
Azhar Dewji
a1ca9025e7
Fix error when trying to call list_schemas macro for BigQuery adapter
2020-05-06 17:10:04 -04:00
Jacob Beck
9cd7cbc9e3
Merge pull request #2409 from fishtown-analytics/fix/plugin-version-files
...
update __version__.py files, add them to .bumpversion.cfg (#2408 )
2020-05-06 10:19:03 -06:00
Jacob Beck
2d3dc1f6cc
update __version__.py files, add them to .bumpversion.cfg
2020-05-06 09:41:10 -06:00
Drew Banin
d01c1614a5
Merge pull request #2407 from fishtown-analytics/fix/dupe-resource-type-error
...
Fix for error in error handler for duped data tests
2020-05-06 11:00:19 -04:00
Drew Banin
fb5320f7bc
( #2406 ) Fix for error in error handler for duped data tests
2020-05-06 10:10:46 -04:00
Jacob Beck
7c916e9bdb
Merge pull request #2321 from snowflakeseitz/column_comments
...
Column comments
2020-05-05 12:23:02 -06:00
Andrew Seitz
b774702ee8
removing duplicates in common
2020-05-05 10:21:36 -07:00
Andrew Seitz
ffaaacc148
adding in changelog
2020-05-05 10:05:14 -07:00
Jacob Beck
9de9335554
Merge pull request #2400 from fishtown-analytics/fix/disabling-tracking-resets-invocation-id
...
Fix: disabling tracking resets invocation ID (#2398 )
2020-05-05 10:25:46 -06:00
Drew Banin
feae21992a
Merge pull request #2399 from fishtown-analytics/docs/0.17.0
...
Update docs site for 0.17.0
2020-05-05 12:23:19 -04:00
Jacob Beck
18cfe81e00
use is not instead of != on time comparison check
2020-05-05 09:41:46 -06:00
Jacob Beck
869bdc454d
Disabling tracking no longer resets the invocation ID
...
- re-use the same active user
- add disable_tracking method that does not reset invocation ID
2020-05-05 09:31:46 -06:00
Drew Banin
6725234466
Fix missing source columns in docs site
2020-05-05 11:08:47 -04:00
Drew Banin
05bf1512f9
( #2284 ) bump docs build
2020-05-05 09:07:21 -04:00
Andrew Seitz
bd2d1f7722
fixing failing test on incremental
2020-05-04 14:49:05 -07:00
Andrew Seitz
f58bc49176
fixing default
2020-05-04 14:49:05 -07:00
Andrew Seitz
5cabafc257
fixing rebase error
2020-05-04 14:49:05 -07:00
Andrew Seitz
0fdff04fc6
fixing rebase
2020-05-04 14:48:55 -07:00
Andrew Seitz
1448d0b0c2
adding in support for views
2020-05-04 14:48:05 -07:00
Andrew Seitz
f843e658a1
fixing spacing
2020-05-04 14:48:05 -07:00
Andrew Seitz
06ffb6dd53
adding in Drews feedback
2020-05-04 14:48:05 -07:00
Andrew Seitz
78110e17e5
basics to get column comments working
2020-05-04 14:48:05 -07:00
Andrew Seitz
74df1a1c31
adding persist docs tests
2020-05-04 14:48:05 -07:00
Andrew Seitz
d9862af8e1
adding in support for views
2020-05-04 14:47:46 -07:00
Andrew Seitz
ea82d8f8ca
fixing spacing
2020-05-04 14:47:46 -07:00
Andrew Seitz
8de3235a63
adding in Drews feedback
2020-05-04 14:47:43 -07:00
Andrew Seitz
eb3ed5744b
basics to get column comments working
2020-05-04 14:47:19 -07:00
Andrew Seitz
b98a4d82c3
rebasing
2020-05-04 14:42:47 -07:00
Jacob Beck
ac562d9388
Add dbt:0.17.0b1 dockerfiles and requirements
2020-05-04 15:09:43 -06:00
Jacob Beck
d0cb960455
Bump version: 0.17.0a1 → 0.17.0b1
2020-05-04 14:59:39 -06:00
Jacob Beck
7db16f8521
Merge pull request #2395 from fishtown-analytics/fix/bigquery-numbery-datasets
...
Add a special as_text filter to the native env (#2384 )
2020-05-04 14:50:01 -06:00
Jacob Beck
229d3986c2
PR feedback: text filters for everyone
2020-05-04 14:17:13 -06:00
Jacob Beck
c237c8ee9e
Merge pull request #2393 from fishtown-analytics/fix/suppress-bigquery-job-sql
...
Suppress the bigquery job sql in exception messages (#2383 )
2020-05-04 13:49:22 -06:00
Jacob Beck
bd9a1320fa
Merge pull request #2349 from franloza/fix/2173
...
Use original path to mimick resource subdirectory structure
2020-05-04 13:46:11 -06:00
Jacob Beck
9eb4506117
Merge branch 'dev/octavius-catto' into fix/2173
2020-05-04 13:45:25 -06:00
Jacob Beck
58c95bb399
Merge pull request #2389 from sethwoodworth/patch-1
...
Adds missing comma to `dbt compile` help text
2020-05-04 13:43:57 -06:00
Jacob Beck
af5af829cf
add missing as_text to sources tests
2020-05-04 13:41:45 -06:00
Jacob Beck
0961bdb506
Add a special as_text filter to the native env that tells dbt to render the value as a string no matter what.
...
Use like "{{ 1991 | as_text }}" -> "1991"
Also, moved the weird special quoting handling logic out and instead avoid calling the renderer on column_name arguments
2020-05-04 13:18:49 -06:00
Jacob Beck
7b9111d386
Merge branch 'dev/octavius-catto' into patch-1
2020-05-04 10:37:39 -06:00
Jacob Beck
f8580a23ff
Suppress the bigquery job sql in exception messages
2020-05-04 10:35:50 -06:00
Drew Banin
b5f99881fe
Merge pull request #2390 from fishtown-analytics/fix/snapshot-staging-is-not-atomic
...
Fix/snapshot staging is not atomic
2020-05-04 12:24:49 -04:00
Seth Woodworth
6266216fda
Merge branch 'dev/octavius-catto' into patch-1
2020-05-04 12:22:43 -04:00
Seth Woodworth
ca39ac5f00
Update changelog with PR #2389
2020-05-04 12:21:25 -04:00
Drew Banin
5d99798c7d
Merge branch 'dev/octavius-catto' into fix/snapshot-staging-is-not-atomic
2020-05-04 11:00:29 -04:00
Drew Banin
24ed84dddd
Merge pull request #2391 from fishtown-analytics/fix/snapshot-use-updated-at-timestamp-for-comparisons
...
Fix for changing snapshot strategy types
2020-05-04 10:39:23 -04:00
Drew Banin
e82e68d22a
update changelog
2020-05-04 10:39:00 -04:00
Jacob Beck
e6bb06030c
Merge pull request #2378 from fishtown-analytics/feature/pg-rs-persist-docs
...
Column and table comments for postgres/redshift (#2333 )
2020-05-04 07:47:23 -06:00
Jacob Beck
595c82c1f1
Merge pull request #2387 from fishtown-analytics/fix/ambiguous-alias-exception
...
fix the ambiguous alias exception to respect databases
2020-05-04 07:18:55 -06:00
Jacob Beck
a46855821b
Merge branch 'dev/octavius-catto' into fix/ambiguous-alias-exception
2020-05-04 07:18:27 -06:00
Drew Banin
89ba3e707a
Tweak code comment
2020-05-04 09:00:20 -04:00
Drew Banin
2cd6a2d7a0
( #2350 ) Fix for changing snapshot strategy types
2020-05-02 12:38:57 -04:00
Drew Banin
1f758f90d0
Merge pull request #2376 from fishtown-analytics/fix/handle-query-array-types
...
Fix/handle query array types
2020-05-02 10:00:14 -04:00
Drew Banin
8681dd8c93
( #1884 ) Fix for non-atomic snapshot staging tables
2020-05-02 09:28:31 -04:00
Drew Banin
0781cef8b1
( #2337 ) Handle array values in agate dataframe building
2020-05-02 09:23:03 -04:00
Seth Woodworth
5edbe5758e
Adds missing comma to dbt compile help text
2020-05-01 13:50:07 -04:00
Jacob Beck
20c5c4c3dd
fix the ambiguous alias exception to respect databases
...
The alias name check is now tied to the behavior of adapter.Relation.create_from(...)
Plugins that override the `include` of their relations will use whatever they render to for the check
the actual exception-raising code gets the name that was compared instead of generating its own
Finally, I added a reasonable fallback behavior since this method was exposed to the context
2020-05-01 10:43:38 -06:00
Jacob Beck
e376c14e8a
Merge branch 'dev/octavius-catto' into fix/2173
2020-05-01 08:45:08 -06:00
Drew Banin
68babfb4bb
Merge pull request #2200 from fishtown-analytics/fix/sql-header-in-incrementals
...
Fix missing sql_header for incremental models
2020-05-01 10:11:59 -04:00
Drew Banin
f3d4377fdd
Merge branch 'dev/octavius-catto' into fix/sql-header-in-incrementals
2020-05-01 10:10:21 -04:00
Fran Lozano
efaeb6786e
Merge remote-tracking branch 'upstream/dev/octavius-catto' into fix/2173
2020-05-01 12:32:44 +02:00
Fran Lozano
96d2978a23
Update CHANGELOG.md
2020-05-01 12:29:37 +02:00
Jacob Beck
6e1665d1fb
Merge pull request #2382 from fishtown-analytics/fix/remove-bq-location
...
remove location and information_schema.schematas query from catalog (#2320 )
2020-04-30 18:47:08 -06:00
Jacob Beck
c6603be194
Handle funky incremental behavior in case existing materializations use it
...
- also fixed the default incremental to include that it is creating a table
2020-04-30 14:49:34 -06:00
Jacob Beck
d17e706351
improve parsing errors, include line:char instead of absolute character index
2020-04-30 14:49:34 -06:00
Jacob Beck
acd978e054
Column and table comments
...
- add some table comment framework stuff
- have redshift/postgres catalogs include table comments
- have redshift/postgres add comments to columns/tables/views
- push some bigquery-specific formatting into bigquery
- add tests for table comments
2020-04-30 14:49:34 -06:00
Jacob Beck
ed6591e450
remove location and information_schema.schematas query from catalog
2020-04-30 13:52:13 -06:00
Fran Lozano
fc61869ea8
Revert "Fix wrong path in test"
...
This reverts commit a3e6518c41 .
2020-04-30 01:24:43 +02:00
Fran Lozano
80501e7f8f
Fix wrong path in test
2020-04-30 01:19:09 +02:00
Fran Lozano
a3e6518c41
Fix wrong path in test
2020-04-30 00:58:19 +02:00
Fran Lozano
b71496a25d
Fix wrong paths in tests
2020-04-30 00:45:55 +02:00
Fran Lozano
391ef70e67
Change build paths in docs generate tests according to new path format
2020-04-30 00:21:20 +02:00
Drew Banin
29e2bbc0c2
Merge branch 'dev/octavius-catto' into fix/sql-header-in-incrementals
2020-04-29 15:52:44 -04:00
Jacob Beck
5884f7df55
Merge pull request #2357 from fishtown-analytics/feature/schema-yml-rendering-3
...
Schema yaml rendering and source overrides
2020-04-29 13:52:22 -06:00
Jacob Beck
6708046951
Merge pull request #2341 from dcereijodo/log-depends-on-extra
...
log 'depends_on_nodes' node attribute in the log record 'extra' field
2020-04-29 07:13:33 -06:00
Jacob Beck
4751a5f3e2
Merge pull request #2361 from Fokko/patch-1
...
Add python_requires to setup.py
2020-04-28 13:17:30 -06:00
Jacob Beck
0b82def5f5
support windows in source overrides tests
2020-04-28 12:49:49 -06:00
Jacob Beck
ee529a5472
fix catalog unit tests
2020-04-28 11:52:03 -06:00
Jacob Beck
d38344125e
move catalog nodes entry into sources
...
Fix tests accordingly
2020-04-28 11:37:00 -06:00
Jacob Beck
caee9415e4
PR feedback
...
- add paths to source patches, apply them to parsed source definitions
- warn on unused source patches
- fix error on duplicate source patches
- add sources back into the catalog
- update changelog
2020-04-28 11:37:00 -06:00
Jacob Beck
7e85ad95c7
Add some very special quoting rules for rendering
...
Handle quoting in render() instead of get_rendered code
- avoid stripping the quotes from a string that was returned as jinja
2020-04-28 11:29:06 -06:00
Jacob Beck
86a35d6c13
Add file paths to dbt_project.yml when it fails to render
2020-04-28 11:29:06 -06:00
Jacob Beck
ff3360afc0
flip the flag
...
Add/modify tests to test native rendering/config application
2020-04-28 11:29:06 -06:00
Jacob Beck
eea322e647
update changelog
2020-04-28 11:29:05 -06:00
Jacob Beck
120f0f0959
Tests
...
add override tests
fix some unit tests
2020-04-28 11:28:48 -06:00
Jacob Beck
38443cf3f5
implement source patching
2020-04-28 11:28:48 -06:00
Jacob Beck
c69f28e0a0
move sources into their own dict
2020-04-28 11:28:48 -06:00
Jacob Beck
704e44e479
add source patches, distinguished by override, store them on run results/the manifest
2020-04-28 11:28:48 -06:00
Jacob Beck
c19a7d7b32
Docs now get the schema.yml parser vars
...
The SchemaParser now renders the entire schema.yml except tests/descriptions
Snapshot config resolution
2020-04-28 11:28:48 -06:00
Jacob Beck
7323ddbabf
Give sources a sane FQN, initial schema var rendering stuff
2020-04-28 11:28:48 -06:00
Jacob Beck
0b7dc326e9
remove docs parse context - not really used
2020-04-28 11:28:48 -06:00
Jacob Beck
afc0341b54
Merge pull request #2363 from fishtown-analytics/fix/vars-invalidation
...
Vars invalidation (#2265 )
2020-04-28 10:06:22 -06:00
Jacob Beck
0df8462562
Merge branch 'dev/octavius-catto' into fix/vars-invalidation
2020-04-28 08:36:47 -06:00
David
305ace528a
Merge branch 'dev/octavius-catto' into log-depends-on-extra
2020-04-28 09:28:45 +02:00
Drew Banin
f7b8e5c861
Merge pull request #2348 from fishtown-analytics/fix/decimal-to-json-bq
...
Fix for non-json-serializable values in BigQuery nested columns
2020-04-27 15:22:16 -04:00
Jacob Beck
8bf5dd0502
Merge pull request #2364 from fishtown-analytics/feature/webbrowser-no-deadlock
...
add --no-browser argument to docs serve
2020-04-27 12:04:19 -06:00
Fokko Driesprong
8d84da40e1
Add python_requires to setup.py
...
This will let people know that they are running
a too old version of Python
Signed-off-by: Fokko Driesprong <fokko@apache.org >
2020-04-27 19:35:05 +02:00
Jacob Beck
ab886cde16
added --no-browser argument to docs serve
2020-04-27 11:09:20 -06:00
Jacob Beck
ab5432daa0
add a test for vars that change manifest-compile behavior
2020-04-27 10:31:21 -06:00
Jacob Beck
60d4708f15
refactor test suite
2020-04-27 10:31:21 -06:00
Jacob Beck
7fba68fcbd
invalidate the manifest when vars change
2020-04-27 10:31:21 -06:00
Fran Lozano
8f45ecebb4
Consider many-to-one relationships from nodes to files
2020-04-26 21:43:01 +02:00
Christian Kohlberg
c51ba50946
Expose Snowflake query id in case of an exception raised by the connector
...
* Allows to search for the query id inside the Snowflake query history
* Allows for better communication with snowflake support through query ids
2020-04-26 13:28:08 +02:00
Jacob Beck
616b32eb67
Merge pull request #2312 from fishtown-analytics/feature/dbt-project-v2
...
dbt project v2
2020-04-22 07:40:57 -06:00
Fran Lozano
278f764d61
Use original path to mimick resource subdirectory structure
2020-04-22 01:18:18 +02:00
Jacob Beck
5ed5693cd5
remove warning for "vars" in config blocks
2020-04-21 16:43:14 -06:00
Jacob Beck
236a087a42
skip deprecation warnings on non-graph runnable tasks so the deprecation warnings on graph runnable tasks are better
2020-04-21 15:10:17 -06:00
Drew Banin
044f8fce4e
Update CHANGELOG.md
...
Co-Authored-By: Jacob Beck <beckjake@users.noreply.github.com >
2020-04-21 16:19:38 -04:00
Jacob Beck
618adf501d
PR feedback: adjust deprecation warning copy
2020-04-21 13:46:39 -06:00
Jacob Beck
0da3b07fb3
add type annotations, add some unit tests for MultiDict
2020-04-21 13:15:59 -06:00
Drew Banin
4bea303c5f
( #2336 ) Fix for non-json-serializable values in BigQuery nested columns
2020-04-21 15:00:04 -04:00
Jacob Beck
c18b72b53d
PR feedback
...
Handle empty vars dict
Check sources for unused configs
- add a test
Warn when dbt finds "vars" in "models", "seeds", etc blocks
- add a test
Clean up enabled/disabled code to share between sources and nodes
- only log downstream tests at debug level when a source is disabled
- include the offending project name in the v1 deprecation message
- Fix tests that care about error messages, add new ones
2020-04-21 12:41:19 -06:00
dcereijodo
ac9ea53e8e
updated changelog and contributors
2020-04-21 08:24:07 +02:00
Jacob Beck
669a18564f
instead of a special "config" key, use a "+" prefix and only require it on dict values
...
Make the package-duplicates test use a local dependency
2020-04-20 14:30:44 -06:00
Jacob Beck
c6b234673c
deprecate v1 configs
...
Bump included projects to v2
fix tests
2020-04-20 11:56:14 -06:00
Jacob Beck
1adf994740
Convert tests to v2, fix them
...
Also add some missing agate methods to stubs
fix tests from splitting the config renderer
2020-04-20 11:56:14 -06:00
Jacob Beck
72d82e45ed
Fix some mypy issues
...
Split up hooks to expose the hook dict
Remove some odd imports
2020-04-20 11:56:14 -06:00
Jacob Beck
070e13ded8
Split ConfigRenderer into parts
...
Docs now get the schema.yml parser vars
The SchemaParser now renders the entire schema.yml except tests/descriptions
2020-04-20 11:56:14 -06:00
Jacob Beck
c7d8031681
Support for v1/v2 config blocks
...
- Backwards compatiblity shims from v2 to v1 configs
- config merging for v2
- compatibility shim for parsing/contexts
- defer var lookups as late as possible
- fixed ContextConfigType to be a proper Union of the two context config types
- Fix adapter configs to be proper dataclasses
- fix unused config paths error
- make v2 config parsing not alter its source data
2020-04-20 11:31:00 -06:00
Jacob Beck
bf8932efaa
Make node configs work like they should, clean up the json schema/parsing for snapshot configs
2020-04-20 11:30:59 -06:00
Jacob Beck
ff9e0cd4cf
Add get_config_class_by_name function for adapters
2020-04-20 11:30:59 -06:00
Jacob Beck
1522d4cb2b
Move configs into their own graph module
...
Configs all get defaults, a base class
Added SourceConfig
Introduced the idea of MergeBehavior and ShowBehavior
Implemented updates/merges based on MergeBehavior for configs
Implemented typed adapter-specific configs and behavior
2020-04-20 11:30:59 -06:00
Jacob Beck
fac8a286e8
Split SourceConfig into LegacyContextConfig and ConfigUpdater
...
Renamed config.config to config.build_config_dict()
mypy annotations
rename a module to avoid duplicating names
2020-04-20 11:18:29 -06:00
Jacob Beck
d37e74c62c
Rework vars to support v1 and v2
...
- Rename project contract to projectv1
- Add load_dependencies and dependencies attrs to RuntimeConfig
- Moved project loading from parser/manifest.py into methods of RuntimeConfig
- Move methods that check for unused configs from Project to RuntimeConfig
- they require the adapter type to be correct
- Rework Var in provider vs base context to handle both styles of Var configuration
2020-04-20 11:18:28 -06:00
Jacob Beck
0a1862e92e
refactor source config
2020-04-20 11:18:28 -06:00
dcereijodo
9dd16045fb
log 'depends_on_nodes' node attribute in the log record 'extra' field
2020-04-18 11:11:49 +02:00
Jacob Beck
c05b45b241
Merge pull request #2339 from nickwu241/fix/0.16.1/UnsetProfileConfig-use-args-project-dir
...
Update UnsetProfileConfig.from_args to use args.project_dir if passed
2020-04-17 14:59:24 -06:00
Nick Wu
2a0f69ac5f
Update test to change out of dir that will be deleted
2020-04-17 10:35:09 -07:00
Nick Wu
d5f50954c0
Add test for running simple dbt commands with --project-dir arg
2020-04-17 09:58:10 -07:00
Nicholas Wu
dda9289d77
Update core/dbt/config/runtime.py
...
Co-Authored-By: Jacob Beck <beckjake@users.noreply.github.com >
2020-04-17 07:55:56 -07:00
Nick Wu
66f9ee0b3e
Updatet CHANGELOD.md with dbt deps fix
2020-04-17 03:22:28 -07:00
Nick Wu
72d2cab517
Update UnsetProfileConfig.from_args docstring
2020-04-17 02:40:45 -07:00
Nick Wu
a376d656a7
Update UnsetProfileConfig.from_args to use args.project_dir if passed
2020-04-17 02:30:30 -07:00
Jacob Beck
204fc25c21
Merge pull request #2291 from fishtown-analytics/feature/test-source-overrides-deps
...
Tests for source overriding (#2264 )
2020-04-16 10:57:01 -06:00
Jacob Beck
bea8c72a82
Merge branch 'dev/octavius-catto' into feature/test-source-overrides-deps
2020-04-16 10:18:13 -06:00
Jacob Beck
16c1fcf4db
Update CHANGELOG.md
...
Co-Authored-By: Drew Banin <drew@fishtownanalytics.com >
2020-04-16 10:17:12 -06:00
Jacob Beck
566f78a95c
Merge pull request #2322 from fishtown-analytics/feature/show-objects-in-schema
...
use "show objects in schema" (#2174 )
2020-04-16 07:26:31 -06:00
Jacob Beck
6ca707822e
Merge branch 'dev/octavius-catto' into feature/show-objects-in-schema
2020-04-16 07:21:18 -06:00
Jacob Beck
59477a32bc
Merge pull request #2324 from fishtown-analytics/feature/sf-describe-columns-in-table
...
Use "describe table" to get the columns in a relation on snowflake (#2260 )
2020-04-16 07:19:12 -06:00
Jacob Beck
bc38750d47
PR feedback, tests
2020-04-15 10:11:30 -06:00
Jacob Beck
51278c0fbf
Merge pull request #2325 from fishtown-analytics/fix/bigquery-include-vs-quote-policy
...
Fix include_policy vs. quote_policy bug in BigQueryRelation (#2188 )
2020-04-15 08:54:09 -06:00
Jacob Beck
644e3e8721
Fix #2188
...
The BigQuery information schema previously used its quote policy as the basis
for a new include policy, rather than its include policy.
2020-04-14 14:42:29 -06:00
Jacob Beck
107bc5c217
Use "describe table" to get the columns in a relation on snowflake
2020-04-14 14:02:27 -06:00
Jacob Beck
1313445afe
change list_relations_without_caching to use "show objects in schema"
2020-04-14 12:33:04 -06:00
Jacob Beck
79db8807a6
Merge pull request #2319 from fishtown-analytics/feature/compile-ls-relation-caches
...
dbt compile and dbt ls relation caches (#1705 )
2020-04-14 12:22:33 -06:00
Drew Banin
4dc12c72c2
Merge pull request #2310 from fishtown-analytics/fix/incorrect-skip-counts
...
Fix incorrect SKIP count in stdout
2020-04-14 13:51:26 -04:00
Jacob Beck
ac427bdc0c
populate_adapter_cache for everyone
...
Added the populate_adapter_cache to the GraphRunnableTask before_run method
- Previously only Run used it
2020-04-14 10:44:25 -06:00
Drew Banin
96a3736cd7
( #2095 ) fix incorrect SKIP count in stdout
2020-04-14 12:30:31 -04:00
Jacob Beck
6424b65097
Merge pull request #2318 from fishtown-analytics/fix/bq-dp-models-failure
...
Fix the tests (#2317 )
2020-04-14 10:21:17 -06:00
Jacob Beck
c6121675bb
re-add quoting logic
2020-04-14 08:07:00 -06:00
Jacob Beck
626f835601
Jinja removed a keyword argument
...
Since they apparently will do this in patch releases, pinnned to precisely 2.11.2
But, it means we don't have to override NativeSandboxTemplate.render
anymore, so that's nice
2020-04-14 07:27:27 -06:00
Jacob Beck
aa38c8101b
Fix the tests
...
This looks like a neat interplay between yaml's anchor/pointer notation (it's
a reference, not a copy!), and dbt inserting 'model' into the keyword args for
tests.
I've added a `copy.deepcopy` call to where dbt collects test arguments from the
underlying file, so the 'model' is inserted into a fresh keyword arguments
dictionary.
2020-04-13 20:18:30 -06:00
Drew Banin
d2760d5a49
Merge pull request #2298 from rodrigodelmonte/fix-diststyle-auto
...
Change redshift diststyle behavior
2020-04-09 21:15:34 -04:00
Rodrigo Monte
0c3f9e7e2e
Merge branch 'dev/octavius-catto' into fix-diststyle-auto
2020-04-09 22:57:34 +02:00
Jacob Beck
fd4a33e450
Fix merge: it doubled up a few lines
2020-04-08 12:03:19 -06:00
Jacob Beck
f07face7c0
Merge branch 'dev/0.16.1' into dev/octavius-catto
2020-04-08 11:47:47 -06:00
Jacob Beck
935f985e25
Merge pull request #2297 from sumanau7/fix/return_proper_msg_for_empty_profile
...
Return proper error message when empty profiles.yml is used for project
2020-04-08 10:44:49 -06:00
Sumanau Sareen
35b43e9708
Update changelog.md with github issue and PR details
2020-04-08 20:57:42 +05:30
Sumanau Sareen
9991b066c7
Check if profile data with given profile name is empty
2020-04-08 20:57:07 +05:30
Sumanau Sareen
c76d6f34a2
Return proper error message when empty profiles.yml is used for dbt project
2020-04-08 20:57:07 +05:30
Jacob Beck
380f7c2e51
Merge pull request #2263 from sumanau7/fix/read_filenames_using_case_insenstive_extension
...
Read filenames using case insensitive extension
2020-04-08 08:02:03 -06:00
rodrigodelmonte
ab07b8ca06
Change redshift diststyle behavior ( #2246 )
...
* Don't provide a diststyle to redshift tables when configured as "auto"
2020-04-07 22:34:04 +02:00
Sumanau Sareen
6f302e26b3
Update changelog.md with github issue and PR details
2020-04-07 22:49:50 +05:30
Sumanau Sareen
f9343c4683
1. Add integration test for yaml schema extension warning. 2. Add integration test for checking case-insensitive extensions
2020-04-07 22:08:11 +05:30
Sumanau Sareen
466ed1f8e6
Add warning in case core config files are found with yaml extension
2020-04-07 21:49:06 +05:30
Sumanau Sareen
83cdd40191
FilesystemSearcher now searches for file names ignoring the case of the extension
2020-04-07 21:49:06 +05:30
Drew Banin
579a6d6064
Update changelog
2020-04-06 20:35:48 -04:00
Drew Banin
3a5eb4c2ad
Fix for bq insert_overwrite incrementals, add additional tests
2020-04-06 17:58:00 -04:00
Drew Banin
40455108bb
Merge branch 'dev/octavius-catto' of github.com:fishtown-analytics/dbt into fix/sql-header-in-incrementals
2020-04-06 17:12:28 -04:00
Jacob Beck
2c241025c9
Merge pull request #2279 from sumanau7/add_dbt_plugin_version
...
Add dbt plugin version
2020-04-06 14:41:00 -06:00
Sumanau Sareen
f5697fd82b
Update CHANGELOG.md
...
Co-Authored-By: Jacob Beck <beckjake@users.noreply.github.com >
2020-04-07 00:55:54 +05:30
Sumanau Sareen
10d20197ed
Update changelog.md with github issue and PR details
2020-04-07 00:38:47 +05:30
Sumanau Sareen
5d5913efeb
Add DBT Plugin information to get_version_information
2020-04-05 00:33:16 +05:30
Jacob Beck
8cc0178b64
Add dbt:0.16.1rc1 dockerfiles and requirements
2020-04-03 12:24:18 -06:00
Jacob Beck
6a26a5f47d
Merge pull request #2258 from fishtown-analytics/feature/model-select-paths
...
Add a `path:` selector to the node selector (#454 )
2020-04-03 12:23:23 -06:00
Jacob Beck
01f5ce73a3
Bump version: 0.16.1a1 → 0.16.1rc1
2020-04-03 12:13:45 -06:00
Jacob Beck
1e94b1f5bb
Merge pull request #2293 from fishtown-analytics/fix/duplicate-plugin-macros
...
Fix an error in exception handling for duplicate macros (#2288 )
2020-04-03 12:12:41 -06:00
Jacob Beck
13e8b0dd0d
Merge branch 'dev/0.16.1' into fix/duplicate-plugin-macros
2020-04-03 12:12:02 -06:00
Jacob Beck
f25b830aad
Merge pull request #2294 from fishtown-analytics/fix/repair-create-adapter-script.py
...
Update the create adapter script to dbt 0.16.1
2020-04-03 11:05:46 -06:00
Jacob Beck
be0e12edb0
Fix an error in exception handling
...
Add a unit test
2020-04-03 10:53:14 -06:00
Jacob Beck
6aafd827f9
Update the create adapter script to dbt 0.16.1
...
Make the base dbt version in the script managed by bumpversion
2020-04-03 10:29:05 -06:00
Jacob Beck
41746eebbb
Tests for source overriding
...
- Make sure dependency sources can be selected from the root project
- Make sure the root project can override dependency sources
2020-04-02 10:34:11 -06:00
Jacob Beck
fb3019f3f1
Merge pull request #2290 from fishtown-analytics/fix/deps-no-profile-required
...
Do not require a profile in "dbt deps" (#2231 )
2020-04-02 10:16:46 -06:00
Jacob Beck
aa3cb37e05
PR feedback: name change
2020-04-02 09:42:22 -06:00
Jacob Beck
f0a4810b08
Use config.credentials.type instead of get_adapter(config).type()
...
- remove import cycle
- works even if adapter class has been loaded but adapter object has not been registered
2020-04-02 09:17:28 -06:00
Jacob Beck
42e8c56b13
in "dbt deps", if a profile cannot be found/loaded, use invalid values instead
...
Add tests
2020-04-02 08:42:16 -06:00
Jacob Beck
ff0e955ccb
Merge pull request #2280 from fishtown-analytics/fix/add-partitions-bq-config
...
Add "partitions" to the adapter specific config (#2256 )
2020-04-01 12:12:49 -06:00
Jacob Beck
f8ee0561cb
PR feedback
...
Also, added more tests
2020-04-01 10:14:24 -06:00
Jacob Beck
6c7d71028b
Add a path: selector to the node selector
...
Instead of always being FQN by default, path-like selectors default to PATH
Paths referring to non-root packages are ignored
Added a test to the dependency tests that checks both of these
2020-04-01 10:14:23 -06:00
Jacob Beck
c63f60ef7c
Add "partitions" to the adapter specific config
...
And tests, of course
2020-04-01 10:06:42 -06:00
Jacob Beck
79e8a86750
Merge pull request #2281 from fishtown-analytics/fix/quoted-information-schema
...
do not double-quote the database name (#2267 )
2020-04-01 10:03:44 -06:00
Jacob Beck
aa0e58ef3b
Merge pull request #2262 from kyleabeauchamp/kab_1995
...
Add db_groups and autocreate flags to Redshift config
2020-04-01 09:53:18 -06:00
Jacob Beck
4c93b51427
Merge pull request #2248 from fishtown-analytics/feature/document-contexts
...
Add documentation for contexts and manifests, plus extraction scripts (#2202 )
2020-04-01 09:30:22 -06:00
Kyle Beauchamp
f4f10f6dd2
Fixes to changelog
2020-04-01 07:58:25 -07:00
Jacob Beck
e9b537a861
Add documentation for contexts and manifests, plus extraction scripts
...
Contexts are generally documented based on the dbt documentation page
Manifest is documented using the 'description' metadata field that hologram supports
Added two scripts:
- one to extract context metadata
- one to extract the manifest/run result/catalog json schemas
- both scripts run during circleCI and dump the json files to disk
- which are then uploaded as build artifacts on circle
2020-04-01 08:46:01 -06:00
Jacob Beck
b385d176a7
Merge pull request #2251 from fishtown-analytics/fix/render-profile-name-base-context
...
Render profile_name in the base context (#2230 )
2020-04-01 08:40:23 -06:00
Jacob Beck
df2ae076b9
Merge branch 'dev/0.16.1' into fix/render-profile-name-base-context
2020-04-01 08:39:19 -06:00
Jacob Beck
9e6e381a40
Merge pull request #2282 from fishtown-analytics/feature/append-query-comment-backport
...
append query comment backport (#2138 )
2020-04-01 08:37:55 -06:00
Jacob Beck
a8bdaba497
Merge branch 'dev/0.16.1' into feature/append-query-comment-backport
2020-04-01 08:37:26 -06:00
Jacob Beck
e3ac64d14a
PR feedback
2020-04-01 08:37:07 -06:00
Jacob Beck
c146e5076e
Merge pull request #2252 from fishtown-analytics/fix/docs-blocks-embedding
...
Make the contents of the raw blocks regex pattern non-greedy (#2241 )
2020-04-01 08:35:24 -06:00
Jacob Beck
374d0b4e91
Merge branch 'dev/0.16.1' into fix/docs-blocks-embedding
2020-04-01 08:34:56 -06:00
Jacob Beck
c415b6b41f
Merge pull request #2237 from fishtown-analytics/feature/slim-manifest
...
slim the manifest
2020-04-01 08:33:48 -06:00
Jacob Beck
3807b0b6cd
CHANGELOG update
2020-03-31 15:14:16 -06:00
ilkin Balkanay
8b05113cc8
pass integration test
...
- to disable query comments use : `'query-comment': ''`
2020-03-31 15:05:10 -06:00
ilkin Balkanay
304b11ee09
NoValue type definition updated.
...
- hologram version updated to 0.0.6 to be able to use QueryComment, NoValue and str within the same Union type.
2020-03-31 15:05:10 -06:00
ilkin Balkanay
11b7bf77c0
updated changelog
...
- added ilkinulas to contributers list
2020-03-31 15:05:10 -06:00
ilkin Balkanay
ee2d990ece
updated change log.
...
- replaced pr link with issue link
2020-03-31 15:05:10 -06:00
ilkin Balkanay
031d844d30
refactored _get_comment_macro method
...
- isinstance check is needed to pass mypy checks
2020-03-31 15:05:10 -06:00
ilkin Balkanay
33d3c9f0bd
pass unit test
2020-03-31 15:05:10 -06:00
ilkin Balkanay
a6aa1c36b3
refactored types (pass mypy checks)
2020-03-31 15:05:10 -06:00
ilkin Balkanay
90542fb499
added QueryComment dataclass
2020-03-31 15:05:10 -06:00
ilkin Balkanay
70031c63b6
updated CHANGELOG
2020-03-31 15:05:09 -06:00
ilkin Balkanay
b781cdad6d
fix integration test.
2020-03-31 15:05:09 -06:00
ilkin Balkanay
504c163570
fixed unittest: default value for query_comment is None
2020-03-31 15:05:09 -06:00
ilkin Balkanay
5f294bc587
handle unset query comment
2020-03-31 15:05:09 -06:00
ilkin Balkanay
b202da7d84
return default comment if config.query_comment is None
2020-03-31 15:05:09 -06:00
ilkin Balkanay
43525d622a
fixed mypy error
2020-03-31 15:05:09 -06:00
ilkin Balkanay
0979c67b63
added unit test for query_headers.py
2020-03-31 15:05:09 -06:00
ilkin Balkanay
90abe7bbe7
pass unit tests
2020-03-31 15:05:09 -06:00
ilkin Balkanay
9c9d6248ae
added query comment append feature (wip)
2020-03-31 15:05:09 -06:00
Jacob Beck
8c7fef47a4
do not double-quote the database name
2020-03-31 14:57:31 -06:00
Jacob Beck
78f1afa959
Merge pull request #2259 from jeremyyeo/fix/warn-exception-sql-compilation
...
Make warn return empty string (#2222 )
2020-03-30 08:10:21 -06:00
Jeremy Yeo
f5d358d3cf
add to changelog
2020-03-28 10:58:56 +13:00
Jeremy Yeo
55cd72cf62
add test
2020-03-28 10:23:30 +13:00
Jeremy Yeo
11d62f6c13
make warn return empty string
2020-03-28 09:57:37 +13:00
Jacob Beck
4182c03258
Merge pull request #2255 from fishtown-analytics/fix/all-null-seed-column-types
...
Fix a ValueError on redshift when an all-null column had a text type …
2020-03-27 11:44:44 -06:00
Jacob Beck
2da96e3169
Fix a ValueError on redshift when an all-null column had a text type specified
...
add unit tests for conversions
2020-03-27 09:40:27 -06:00
Jacob Beck
8e3c95b48c
Make the contents of the raw blocks regex pattern non-greedy
...
This fixes an issue where a file with multiple raw blocks got treated as one _big_ one
- it extended from the first {% raw %} to the last {% endraw %}
2020-03-26 13:36:45 -06:00
Jacob Beck
3c3f57ece1
fix unit tests
2020-03-26 13:03:43 -06:00
Jacob Beck
69ec5fd82e
Render profile_name in the base context
2020-03-26 12:44:06 -06:00
Jacob Beck
386895f800
Merge pull request #2249 from fishtown-analytics/fix/rev-google-lower-bound
...
rev the bigquery minimum versions to a big number (#2233 )
2020-03-26 11:42:38 -06:00
Jacob Beck
6065237de4
fix unit test bug that only appears on final release versions
2020-03-26 10:58:57 -06:00
Jacob Beck
a7617745e2
rev the bigquery minimum versions to a big number
2020-03-26 10:57:05 -06:00
Jacob Beck
923f28225a
Bump version: 0.16.0 → 0.16.1a1
2020-03-26 10:57:05 -06:00
Jacob Beck
7dfef27886
Slim the manifest
...
remove raw_sql from ParsedMacros, remove file_contents from docs
get rid of seed_file_path
Do not write 'manifest.files' to disk
2020-03-26 10:29:56 -06:00
Jacob Beck
5c698908a4
Merge pull request #2244 from fishtown-analytics/feature/source-aliases
...
Add support for source aliases (#2133 )
2020-03-26 08:12:17 -06:00
Jacob Beck
d88089fb24
Merge branch 'dev/octavius-catto' into feature/source-aliases
2020-03-25 14:52:11 -06:00
Jacob Beck
0188d03d1b
Merge pull request #2238 from fishtown-analytics/fix/macro-kwarg-error-add-file
...
fix commit messages by using the stack properly (#2073 )
2020-03-25 14:41:10 -06:00
Jacob Beck
7b6ea338ed
Merge pull request #2220 from fishtown-analytics/feature/schema-tests-pass-rendered
...
In schema tests, pass rendered items (#2149 )
2020-03-25 14:40:57 -06:00
Jacob Beck
40e88ded25
Merge pull request #2245 from fishtown-analytics/fix/circleci-pipelines
...
set the circleci config version to 2.1 (#2207 )
2020-03-25 14:03:33 -06:00
Jacob Beck
38bcc2b736
override NativeSandboxTemplate.render(), remove the quotes hack
2020-03-25 14:03:17 -06:00
Jacob Beck
73e6941817
circleci 2.1
2020-03-25 13:32:56 -06:00
Jacob Beck
1268c119b5
require jinja 2.11.x
2020-03-25 10:46:45 -06:00
Jacob Beck
434edce8a7
remove extra methods
2020-03-25 10:46:44 -06:00
Jacob Beck
c0ceaee122
Add support for source aliases
2020-03-25 10:43:11 -06:00
Jacob Beck
e9a053c18d
Update core/dbt/node_runners.py
...
Co-Authored-By: Drew Banin <drew@fishtownanalytics.com >
2020-03-25 07:30:02 -06:00
Jacob Beck
427f3bf163
fix commit messages by using the stack properly
2020-03-24 15:12:52 -06:00
Jacob Beck
96cfc49cb9
rev hologram to 0.0.7
2020-03-24 09:13:00 -06:00
Jacob Beck
dc65118f17
Add metadata to tests, add a native env
...
Render test arguments in the native env and pass them along to the context
added/fixed tests
Update changelog
2020-03-24 09:11:24 -06:00
Jacob Beck
31025a0a7f
remove wrapped_sql, move data test wrapping into the node runner
2020-03-24 09:09:49 -06:00
Jacob Beck
d20c301dcd
split test types into data/schema tests
2020-03-24 09:09:49 -06:00
Jacob Beck
0a382c411f
Merge branch 'dev/barbara-gittings' into dev/octavius-catto
2020-03-24 09:09:13 -06:00
Jacob Beck
367bf6849b
Merge pull request #2228 from Raalsky/bug/registry-timeout
...
(#2195 ) Added timeout to registry download call
2020-03-24 08:58:11 -06:00
Jacob Beck
848716607c
Merge pull request #2224 from Raalsky/fail-fast
...
Add --fail-fast argument for dbt run and dbt test
2020-03-24 08:36:45 -06:00
Jacob Beck
0a0e71a9ff
Merge branch 'dev/octavius-catto' into fail-fast
2020-03-24 08:13:24 -06:00
Jacob Beck
ca232dbdf4
Merge branch 'feature/rpc-get-manifest' into dev/octavius-catto
2020-03-24 07:22:58 -06:00
Jacob Beck
efcf78b348
Merge pull request #2199 from ilkinulas/query_comment_append
...
Append query comment
2020-03-24 06:51:14 -06:00
Jacob Beck
bbc0d30fbf
When a user calls the 'get-manifest' API call, compile all the full project and return the compiled manifest from memory.
...
'get-manifest' returns an async ID, just like the existing compile/run/etc
2020-03-23 12:59:36 -06:00
Raalsky
1c60882525
( #2195 ) DBT_HTTP_TIMEOUT moved inside system download call
2020-03-23 17:56:37 +01:00
Raalsky
d6cb415f76
( #2195 ) DBT_HTTP_TIMEOUT environment variable added
2020-03-23 17:48:54 +01:00
Jacob Beck
f58bf878d5
Add dbt:0.16.0 dockerfiles and requirements
2020-03-23 08:20:20 -06:00
Jacob Beck
2fbc716615
Bump version: 0.16.0rc4 → 0.16.0
2020-03-23 07:59:37 -06:00
Raalsky
cce99b1ff2
( #2195 ) Added entry to CHANGELOG
2020-03-23 14:14:21 +01:00
Raalsky
f98b1669fa
( #2195 ) Added timeout to registry download call
2020-03-23 13:35:31 +01:00
Raalsky
efebabe44f
058_fail_fast test method renamed
2020-03-22 18:39:43 +01:00
Rafal Jankowski
42a0461f57
Added integration test 058_fail_fast
2020-03-22 18:27:55 +01:00
Rafal Jankowski
a23ad3ad4a
Linker some_task_done method lock refactor to be more consistent
2020-03-22 18:27:47 +01:00
Rafal Jankowski
1478be716d
FailFastException refactor to RuntimeException
2020-03-22 18:27:32 +01:00
ilkin Balkanay
0eed51dfdd
pass integration test
...
- to disable query comments use : `'query-comment': ''`
2020-03-21 21:36:45 +03:00
İlkin Balkanay
574d65ab4b
Merge branch 'dev/octavius-catto' into query_comment_append
2020-03-21 17:31:30 +03:00
ilkin Balkanay
ac336b20ca
NoValue type definition updated.
...
- hologram version updated to 0.0.6 to be able to use QueryComment, NoValue and str within the same Union type.
2020-03-21 17:21:24 +03:00
Rafal Jankowski
c9613016e2
Added --fail-fast argument for dbt run and dbt test
2020-03-20 19:26:17 +00:00
Jacob Beck
f9696a85d3
Merge pull request #2226 from fishtown-analytics/fix/add-docker-stuff-unlink-dbt
...
Build script updates - add docker stuff, unlink dbt (#2211 )
2020-03-20 13:16:26 -06:00
Jacob Beck
deb97546c0
add 0.16.0rc4 dockerfiles and requirements
2020-03-20 12:16:05 -06:00
Jacob Beck
cea964910d
Also unlink "dbt" if it is installed when test-installing the new homebrew package
2020-03-20 12:11:54 -06:00
Jacob Beck
f8b99f2e34
Bump version: 0.16.0rc3 → 0.16.0rc4
2020-03-20 11:31:17 -06:00
Jacob Beck
3a7e328c6e
Update changelog
2020-03-20 11:31:01 -06:00
Jacob Beck
732792fc80
Merge pull request #2225 from fishtown-analytics/fix/build-script
...
Do not run brew audit on the default formula, it complains
2020-03-20 11:30:18 -06:00
Jacob Beck
dd842ac3b8
Do not run brew audit on the default formula, it complains about versions
2020-03-20 11:15:11 -06:00
Jacob Beck
bdcf10e209
Merge branch 'dev/barbara-gittings' into dev/octavius-catto
2020-03-20 11:05:55 -06:00
Jacob Beck
a43edd9dd8
Merge pull request #2223 from dmateusp/add_dockerize
...
Add Dockerize to test image
2020-03-20 09:34:00 -06:00
ilkin Balkanay
65d9c187b8
updated changelog
...
- added ilkinulas to contributers list
2020-03-20 00:04:14 +03:00
ilkin Balkanay
75e1c1dac8
updated change log.
...
- replaced pr link with issue link
2020-03-19 23:47:06 +03:00
Daniel Mateus Pires
28ccdcc52d
Add Dockerize
2020-03-19 20:36:32 +00:00
ilkin Balkanay
7c34ff166a
refactored _get_comment_macro method
...
- isinstance check is needed to pass mypy checks
2020-03-19 17:48:26 +03:00
ilkin Balkanay
36bf479115
pass unit test
2020-03-19 14:10:19 +03:00
ilkin Balkanay
3b90bfcdc3
refactored types (pass mypy checks)
2020-03-19 11:30:32 +03:00
ilkin Balkanay
ab19135ca6
added QueryComment dataclass
2020-03-19 10:25:29 +03:00
Jacob Beck
41f7b8cb9d
Merge pull request #2215 from fishtown-analytics/feature/add-spark-deps
...
add spark deps to the test container
2020-03-18 18:05:00 -06:00
Jacob Beck
d8b9541b86
Merge pull request #2214 from fishtown-analytics/fix/bigquery-version
...
increase the google-cloud-bigquery lower bound to ensure we support "range_partitioning"
2020-03-18 15:53:57 -06:00
Jacob Beck
a4620adbce
add spark deps
2020-03-18 15:52:23 -06:00
Jacob Beck
02ac87d160
increase the lower bound to ensure we support "range_partitioning"
2020-03-18 15:20:50 -06:00
Jacob Beck
f4c62726ee
Merge pull request #2198 from fishtown-analytics/rework/incremental-overwrite
...
Rework insert_overwrite incremental strategy
2020-03-18 13:11:06 -06:00
Jacob Beck
0844be5a28
Merge pull request #2208 from fishtown-analytics/fix/bigquery-numeric-catalog
...
Fix issue with table databases/schemas/names becoming numbers (#2206 )
2020-03-17 14:41:35 -06:00
Jacob Beck
3a77626cd7
Fix issue with table databases/schemas/names not being interpreted as strings if they look like numbers
2020-03-17 13:07:22 -06:00
Jeremy Cohen
4fee33c020
Unquote partition values in predicate
2020-03-16 13:01:27 -04:00
ilkin Balkanay
e3c53ee874
updated CHANGELOG
2020-03-14 13:55:02 +03:00
Drew Banin
367eff77fc
( #2136 ) Fix missing sql_header for incremental models
2020-03-12 15:56:43 -04:00
Jeremy Cohen
1bed6a1b96
Rename dbt_partitions_for_upsert to dbt_partitions_for_replacement
2020-03-12 10:51:09 -04:00
ilkin Balkanay
1cf9220380
fix integration test.
2020-03-12 17:18:34 +03:00
ilkin Balkanay
fd8629a6cf
fixed unittest: default value for query_comment is None
2020-03-12 16:59:30 +03:00
Jeremy Cohen
be1ca971be
Rm failing irrelevant tint tests
2020-03-12 09:53:15 -04:00
ilkin Balkanay
5fa59d86e7
handle unset query comment
2020-03-12 16:45:36 +03:00
ilkin Balkanay
be80009aca
return default comment if config.query_comment is None
2020-03-12 16:34:30 +03:00
ilkin Balkanay
f18f6af5e0
fixed mypy error
2020-03-12 16:25:58 +03:00
ilkin Balkanay
65f14723ca
added unit test for query_headers.py
2020-03-12 15:46:24 +03:00
ilkin Balkanay
6910847c71
pass unit tests
2020-03-12 13:35:32 +03:00
ilkin Balkanay
62b19b53f9
added query comment append feature (wip)
2020-03-12 10:28:27 +03:00
Jeremy Cohen
8fb6592c6a
Rework insert_overwrite incremental strategy
2020-03-11 22:21:11 -04:00
Jacob Beck
cda35e2a9a
Bump version: 0.16.0rc2 → 0.16.0rc3
2020-03-11 09:20:06 -06:00
Jacob Beck
4472719362
fix twine check
2020-03-11 09:20:02 -06:00
Jacob Beck
6c12b7a951
Update changelog for rc3
2020-03-11 09:07:35 -06:00
Jacob Beck
ad3b63ed5d
Merge pull request #2187 from fishtown-analytics/fix/no-creating-quoted-schemas
...
Do not create already-existing schemas (#2186 )
2020-03-10 22:00:38 -06:00
Jacob Beck
e7545ad183
handle null database/schema issues
2020-03-10 21:29:11 -06:00
Jacob Beck
e390595551
when creating missing schemas, list the quoted database form, but use the unquoted for comparisons
...
add a test
2020-03-06 15:08:33 -07:00
Jacob Beck
b77b3a4566
Merge pull request #2184 from fishtown-analytics/fix/ugly-errors-undefined-values
...
For undefined variables, explicitly raise a useful error on pickling
2020-03-06 13:26:41 -07:00
Jacob Beck
16692e2e5f
tests
2020-03-06 08:41:26 -07:00
Jacob Beck
757bdf69dc
For undefined variables, explicitly raise a useful error on pickling
2020-03-06 08:31:39 -07:00
Jacob Beck
a259f154da
Bump version: 0.16.0b3 → 0.17.0a1
2020-03-06 08:25:03 -07:00
Jacob Beck
e441c6dfa7
Merge pull request #2119 from fishtown-analytics/feature/pkg-build-script
...
Feature: package build script + Docker build script
2020-03-06 08:04:16 -07:00
Jacob Beck
c4c7af88c0
PR feedback + discovered small issues
...
run twine check before uploading
don't install netcat curl ssh on the docker image
skip the dbt resource block initial line when parsing the poet output
unlink existing versions so 'dbt install Formula/...' actually works
Wait a bit for pypi to settle down after uploading to pypi, add retries
2020-03-04 18:52:39 -07:00
Jacob Beck
d78b249827
Fix an issue in 0.16.0b3 where files were erroneously included
...
Previous release files that didn't exist in the new version were ending up in
the wheel (but not source dist). Attempt to fix that by removing build/
directories when we remove dist/ directories
2020-03-04 18:52:39 -07:00
Jacob Beck
af8cdedc07
Build docker images + requirements.txt files
...
some enhancements
2020-03-04 18:52:39 -07:00
Jacob Beck
fc7a62b650
add an initial pass at a dbt package build script
...
Bumps the version (including commit)
builds pypi packages
uploads them to test pypi + prompts for enter/ctrl+c
uploads them to regular pypi
builds homebrew package
overwrites the default homebrew package, if you want
makes appropriate commits
Has a bunch of flags
Requires python 3.8 (homebrew's python version)
Requires .pypirc w/creds
Does not push, so no gh creds required (should it?)
appease homebrew audit:
"version" is supplied by the url
dependencies must be alphabetized
whitespace/newline stuff
2020-03-04 18:52:39 -07:00
Jacob Beck
7919c90b81
Bump version: 0.16.0rc1 → 0.16.0rc2
2020-03-04 16:39:43 -07:00
Jacob Beck
1854d20566
Merge pull request #2181 from fishtown-analytics/fix/pin-another-dang-dependency
...
Pin ciffi to snowflake's versions
2020-03-04 16:29:57 -07:00
Jacob Beck
c5c5c3d863
cffi released a new version, breaking snowflake as a dependency due to version conflicts
2020-03-04 15:58:06 -07:00
Jacob Beck
6c47ae1d88
Bump version: 0.16.0b3 → 0.16.0rc1
2020-03-04 15:27:00 -07:00
Jacob Beck
70ad200dca
Set release version/date in changelog
2020-03-04 15:26:33 -07:00
Drew Banin
341956768a
Merge pull request #2153 from fishtown-analytics/feature/bq-insert-overwrite
...
Feature/bq incremental strategy insert_overwrite
2020-03-04 16:56:42 -05:00
Drew Banin
0656477131
Merge branch 'dev/barbara-gittings' of github.com:fishtown-analytics/dbt into feature/bq-insert-overwrite
2020-03-04 13:45:11 -05:00
Jeremy Cohen
e878d0e76e
Add insert_overwrite as incremental strategy on BQ
2020-03-04 13:43:56 -05:00
Drew Banin
dc6a38c360
Merge pull request #2148 from fishtown-analytics/0.16.0-changelog-updates
...
Update CHANGELOG.md
2020-03-04 12:57:40 -05:00
Drew Banin
f883e11c7a
Merge branch 'dev/barbara-gittings' into 0.16.0-changelog-updates
2020-03-04 12:57:02 -05:00
Drew Banin
7e4402b128
Merge pull request #2179 from fishtown-analytics/0.16.0-docs-site-2
...
Upgrade docs site to support 0.16.0 functionality
2020-03-04 12:54:41 -05:00
Drew Banin
e045e14522
Upgrade docs site to support 0.16.0 functionality
2020-03-04 11:59:31 -05:00
Jacob Beck
f209e17215
Merge pull request #2171 from fishtown-analytics/feature/use-show-schemas-snowflake
...
use show schemas for snowflake list_schemas (#2166 )
2020-03-02 21:27:49 -07:00
Drew Banin
07461d24e0
Merge branch 'dev/barbara-gittings' of github.com:fishtown-analytics/dbt into 0.16.0-changelog-updates
2020-03-02 15:34:09 -05:00
Jacob Beck
47cef1d907
PR feedback: Add exception handler around macro execution, add a message about what is going on when list_schemas fails
2020-03-01 20:48:15 -07:00
Jacob Beck
c59adc3369
strip out the database qutoes on bigquery for the API
2020-02-28 12:04:31 -07:00
Jacob Beck
cc3ba20ec9
refactor, fix flake8/mypy
2020-02-28 11:26:34 -07:00
Jacob Beck
562f3d0bb7
use show schemas for snowflake list_schemas
...
Also fix up the changelog to better reflect releases
2020-02-28 10:30:57 -07:00
Jacob Beck
e8d321cbab
Merge pull request #2164 from mhmcdonald/feature/snowflake_key
...
removing the requirement to have a passphrase on a Snowflake key
2020-02-27 18:36:51 -07:00
Mark McDonald
1edffac2bc
Merge branch 'dev/barbara-gittings' into feature/snowflake_key
2020-02-27 16:05:29 -06:00
Jacob Beck
735ffb3c92
Merge pull request #2154 from dholleran-lendico/dev/barbara-gittings
...
Feature: adding optional sslmode parameter to postgres connection
2020-02-27 13:12:25 -07:00
Mark McDonald
66b60d860e
adding myself to contributors
2020-02-27 11:40:09 -06:00
Mark McDonald
f99d31f7c5
Update CHANGELOG.md
...
Co-Authored-By: Jacob Beck <beckjake@users.noreply.github.com >
2020-02-27 11:34:06 -06:00
dholleran-lendico
170b1d80b5
Update plugins/postgres/dbt/adapters/postgres/connections.py
...
Co-Authored-By: Jacob Beck <beckjake@users.noreply.github.com >
2020-02-27 18:24:03 +01:00
dholleran-lendico
10846653f0
Merge branch 'dev/barbara-gittings' into dev/barbara-gittings
2020-02-27 18:21:31 +01:00
Jacob Beck
5d4d4f32d2
Merge pull request #2157 from fishtown-analytics/feature/threadpool-startup
...
Use threadpools for filling the cache and listing schemas (#2127 )
2020-02-27 09:52:00 -07:00
Mark McDonald
1f49150d30
updating changelog
2020-02-27 09:37:36 -06:00
Mark McDonald
3f583bd927
removing the requirement to have a passphrase on a Snowflake key
2020-02-27 09:31:01 -06:00
Jacob Beck
d11987a2ad
Bump version: 0.16.0b2 → 0.16.0b3
2020-02-26 16:43:26 -07:00
Jacob Beck
79f7985784
Update CHANGELOG.md with release date
2020-02-26 16:43:18 -07:00
Jacob Beck
104422eed9
Bump version: 0.16.0b1 → 0.16.0b2
2020-02-26 16:39:00 -07:00
Jacob Beck
d77e4205c1
un-ignore
2020-02-26 16:38:01 -07:00
Jacob Beck
847bc0995b
Merge pull request #2160 from fishtown-analytics/feature/expanded-schema-search-paths
...
Allow search in macro and analysis paths (#2155 )
2020-02-26 16:31:50 -07:00
Jacob Beck
529b053620
Update CHANGELOG.md
...
Co-Authored-By: Drew Banin <drew@fishtownanalytics.com >
2020-02-26 15:01:01 -07:00
Jacob Beck
4d72e4e257
integration tests fixups
2020-02-26 15:01:01 -07:00
Jacob Beck
8e18af5912
fix unit tests
2020-02-26 15:01:01 -07:00
Jacob Beck
bee5bb8660
Allow search in macro and analysis paths
...
add tests
2020-02-26 15:01:01 -07:00
Jacob Beck
e486872d81
Merge pull request #2163 from fishtown-analytics/fix/broken-merge-0-15-3
...
actually update the connection
2020-02-26 14:57:29 -07:00
Jacob Beck
6e8d4a6fa6
actually update the connection
2020-02-26 10:37:12 -07:00
Jacob Beck
9081303a97
Merge branch 'dev/0.15.3' into dev/barbara-gittings
2020-02-26 10:08:44 -07:00
Jeremy Cohen
40c07afd85
Merge pull request #2156 from fishtown-analytics/fix/bq-range-part-match
...
Bug: fixup bq _partitions_match for range partitions
2020-02-25 19:01:12 -05:00
Jacob Beck
da46a679f3
flake8
2020-02-25 15:52:12 -07:00
Jeremy Cohen
283405dc17
Add test for bq part int<>date switch
2020-02-25 14:41:19 -05:00
Jacob Beck
ed38cbbc0b
Use threadpools for filling the cache and listing schemas
2020-02-25 11:43:07 -07:00
Daniel Holleran
ad91636c74
updating CHANGELOG
2020-02-25 12:10:15 +01:00
Daniel Holleran
5543dab97c
uncommenting sslmode in postgres _coonection_keys
2020-02-25 11:52:37 +01:00
Daniel Holleran
d4ae9a89df
adding 'sslmode' parameter to postgres connection
2020-02-25 11:49:06 +01:00
Jeremy Cohen
032faf2dce
Fixup range_partitioning comparison logic
2020-02-25 01:33:00 -05:00
Jacob Beck
bcea7cc8ad
Merge pull request #2140 from fishtown-analytics/feature/cost-effective-bq-incremental-followup
...
Feature/cost effective bq incremental followup
2020-02-24 10:32:05 -07:00
Jacob Beck
0ad94f80cd
Merge pull request #2151 from fishtown-analytics/fix/more-requirements-changes
...
Dependency revs/bounding (#2147 )
2020-02-24 10:31:27 -07:00
Jacob Beck
7e9533afc7
Merge pull request #2143 from fishtown-analytics/feature/generate-database-name
...
add generate_database_name macro (#1695 )
2020-02-24 10:31:08 -07:00
Jacob Beck
6ca6233387
PR feedback: Changelog update
2020-02-24 08:39:47 -07:00
Jacob Beck
503050904c
add generate_database_name macro
...
Remove old deprecation tests
Add new tests
Fix all the integration tests that do manifest things
2020-02-24 08:39:24 -07:00
Jacob Beck
ba2f9d5781
Dependency revs/bounding
...
remove jinja._compat calls
set an upper bound on jinja anyway
set bound on idna/requests to match snowlfake
update snowflake-connector-python
2020-02-24 08:38:32 -07:00
Jacob Beck
e571cbaeec
Merge pull request #2150 from fishtown-analytics/fix/source-test-arg-rendering
...
Fix source test arg rendering (#2114 )
2020-02-24 08:33:36 -07:00
Jacob Beck
202f8a1678
Update plugins/bigquery/dbt/adapters/bigquery/impl.py
...
Co-Authored-By: Drew Banin <drew@fishtownanalytics.com >
2020-02-24 08:07:31 -07:00
Jacob Beck
4068c6694e
fix dict/PartitionConfig confusion
...
Also, make tests run
2020-02-21 13:36:47 -07:00
Jacob Beck
7d43037d4e
fix tests and types, enhance error message for bad partition configs
2020-02-21 13:00:38 -07:00
Jacob Beck
d6bdb466ae
Actually git add tests
...
Update changelog further to reflect that this is breaking
2020-02-21 11:48:18 -07:00
Jacob Beck
1114885429
Do not render source test arguments during schema.yml parsing
...
Remove extra jinja rendering pass on wrapped_sql
2020-02-21 11:08:23 -07:00
Drew Banin
e29078e5d1
wip
2020-02-21 12:49:53 -05:00
Drew Banin
2bd56de934
push some gnarly logic out of jinja and into python
2020-02-19 21:33:09 -05:00
Drew Banin
df82cd5aec
fix tests
2020-02-19 21:11:42 -05:00
Drew Banin
67571f4ede
quiet bq error logs; refactor to use dataclasses
2020-02-19 18:07:29 -05:00
Drew Banin
fa22c5b071
Merge branch 'dev/barbara-gittings' of github.com:fishtown-analytics/dbt into feature/cost-effective-bq-incremental-followup
2020-02-19 16:34:20 -05:00
Drew Banin
5b7c81eb2d
Update CHANGELOG.md
2020-02-19 14:53:53 -05:00
Jacob Beck
8b20136571
Bump version: 0.15.3a1 → 0.15.3rc1
2020-02-19 12:38:01 -07:00
Jacob Beck
2a426edfe5
Set release date in changelog
2020-02-19 12:37:56 -07:00
Jacob Beck
5dd9c997e2
Merge pull request #2141 from fishtown-analytics/feature/snowflake-refresh-token
...
Support refresh tokens (#2126 )
2020-02-19 12:11:02 -07:00
Jacob Beck
be5fa32c0c
PR feedback: make oauth test skippable
2020-02-19 10:54:21 -07:00
Jacob Beck
0c3fb9528a
Merge pull request #2146 from fishtown-analytics/feature/bigquery-user-agent
...
add a user agent with version to the bigquery client info (#2121 )
2020-02-18 12:42:02 -07:00
Jacob Beck
d5c24b0c63
Support refresh tokens
...
Add a little webserver that spits out a refresh token
Add tests + notes on test support
2020-02-18 10:44:45 -07:00
Jacob Beck
9b8e06c51d
Bump version: 0.15.2 → 0.15.3a1
2020-02-18 10:42:48 -07:00
Jacob Beck
ea4948ff8a
Merge pull request #2120 from nchammas/dbt-debug-empty-project
...
Correctly handle an empty dbt_project.yml in `dbt debug`
2020-02-18 10:39:00 -07:00
Nicholas Chammas
5dc939355a
handle empty dbt_project
2020-02-18 11:32:06 -05:00
Jacob Beck
2672859ee0
add a user agent with version to the bigquery client info
...
Updated unit tests
2020-02-18 08:46:33 -07:00
Jacob Beck
542a65323f
Merge pull request #2124 from fishtown-analytics/fix/requirements-update
...
rev boto3/botocore/snowflake-connector-python/google libraries (#2102 )
2020-02-18 08:00:45 -07:00
Jacob Beck
85e6d7f649
Merge pull request #2125 from fishtown-analytics/feature/macro-sql
...
add macro_sql field (#2118 )
2020-02-17 07:52:59 -07:00
Drew Banin
e73e276e2b
rm semicolons
2020-02-16 18:16:09 -05:00
Drew Banin
1879fb5109
use not-real temp tables on bigquery, fix snapshots
2020-02-16 17:55:30 -05:00
Drew Banin
15cac5afc4
add tests
2020-02-15 18:25:54 -05:00
Drew Banin
1ae17d5745
report bytes billed for scripts, add _dbt_max_partition field
2020-02-15 16:31:56 -05:00
Jeremy Cohen
7106a7c1d1
Implement partition-aware BigQuery merge statements
2020-02-15 15:45:28 -05:00
Drew Banin
0115e469c1
Merge pull request #2130 from fishtown-analytics/fix/add-changelog-0160b1
...
changelog updates, PR template
2020-02-15 13:48:42 -05:00
Drew Banin
7729295bce
Merge pull request #2131 from aaronsteers/aaronsteers/clickable-url
...
Clickable doc serve URL
2020-02-15 13:46:36 -05:00
Drew Banin
3265825446
Merge pull request #2137 from shooka/postgres-role-scope
...
Support Postgres role scope
2020-02-15 13:45:17 -05:00
Nicolai Østby
90fdc9b2e2
Test
2020-02-14 17:51:34 +01:00
Nicolai Østby
9853951aa5
Support act-as in postgres
2020-02-14 17:41:55 +01:00
Jacob Beck
b839c79144
Merge pull request #2135 from fishtown-analytics/fix/rev-dockerfile-pin-stuff
...
fix a bunch of issues caused by virtualenv 20, tox 3.14, six, and ubuntu (#2128 )
2020-02-13 12:54:39 -07:00
Jacob Beck
fad8813911
PR feedbck, fix python_bin
2020-02-13 11:52:34 -07:00
Aaron Steers
ed57876068
adding space
2020-02-13 10:34:09 -08:00
Jacob Beck
5143dbb167
broke building, oops
2020-02-13 10:56:40 -07:00
Jacob Beck
75126aa167
Fix a bunch of issues caused by virtualenv 20, tox 3.14, six, and ubuntu
...
Pin circle to the newest version of the test container
Have the dockerfile also install pinned versions of things
update dev_requirements.txt to match the dockerfile changes
Add python 3.9 while I am in here
2020-02-13 09:59:14 -07:00
Aaron Steers
00a1840a46
remove '.' from after url
2020-02-12 14:12:28 -08:00
Jacob Beck
127f8768aa
changelog updates, PR template
2020-02-12 12:37:47 -07:00
Jacob Beck
6e330fa993
add macro_sql field
...
Add the adapter type to the manifest metadata field.
2020-02-12 11:15:56 -07:00
Jacob Beck
c6ab6459fb
rev boto3/botocore/snowflake-connector-python/google libraries
2020-02-12 08:13:29 -07:00
Jacob Beck
4e58589afd
Merge pull request #2080 from bubbomb/update-docs-duplicates
...
fix docstring duplication error issue #2054
2020-02-11 11:09:05 -07:00
Jacob Beck
89c65affea
Merge pull request #2107 from fishtown-analytics/feature/node-docs-show
...
add a "docs" field to models (#1671 )
2020-02-11 11:04:49 -07:00
Jacob Beck
099b487560
Merge pull request #2104 from fishtown-analytics/fix/rpc-cli-vars-arguments
...
Attach cli vars to configs when they come in via the RPC server (#2092 )
2020-02-11 07:14:42 -07:00
Drew Banin
25235a4d5a
Merge pull request #2078 from sonac/packages-err-msg
...
making error msg for malformed packages.yml better
2020-02-10 23:39:38 -05:00
Jacob Beck
1d329e0364
Bump version: 0.16.0a2 → 0.16.0b1
2020-02-10 15:17:01 -07:00
Jacob Beck
d07f23395c
add a "docs" field to models, parsed from schema.yml. it contains a boolean "show" which defaults to True.
...
Clean up the contracts tests a bit
2020-02-10 14:37:14 -07:00
Jacob Beck
5e3eba6662
Merge pull request #2106 from fishtown-analytics/feature/column-quoting
...
add column-level quoting for tests (#2047 )
2020-02-10 13:58:14 -07:00
Jacob Beck
61adfe42ff
PR feedback
2020-02-10 13:57:45 -07:00
Jacob Beck
80e7e71526
Merge pull request #2105 from fishtown-analytics/fix/docs-generate-bad-docs
...
Fix docstring for dbt docs/source to not include misleading flags (#2038 )
2020-02-10 13:55:04 -07:00
Jacob Beck
1e6790ab05
Merge pull request #2103 from fishtown-analytics/feature/macro-depends-on
...
Feature: node depends_on.macros (#2082 )
2020-02-10 13:39:16 -07:00
Jacob Beck
18358db467
Merge pull request #2097 from fishtown-analytics/feature/ci-builds
...
CI stuff
2020-02-10 13:23:07 -07:00
Jacob Beck
2eea89a0a6
Merge pull request #2099 from heisencoder/fix/debug-log-format
...
Update DEBUG_LOG_FORMAT to correctly zero-pad the microsecond portion
2020-02-10 11:00:05 -07:00
Jacob Beck
dcce90bcc4
Merge pull request #2096 from fishtown-analytics/feature/document-macro-args
...
add macro argument definitions (#2081 , #2083 )
2020-02-10 10:51:04 -07:00
Jacob Beck
b5fc1ef4a3
CI stuff
...
circle: Run unit tests + postgres on circle for outside contributors automatically
circle+azure: skip non-postgres integration tests on for outside contributors
circle+azure: build wheels after tests pass
Update the docker image so venvs can pip install -U pip
2020-02-10 09:35:54 -07:00
Brayden Connole
4ae129d565
Fix broken test
2020-02-07 15:40:43 -07:00
Jacob Beck
3ec9d358f7
add column-level quoting for tests
...
add support in columns for a quote field
add support in source quoting for a column field
add support in models for a quote_columns field
add tests
2020-02-07 14:10:16 -07:00
Jacob Beck
1fc4718d36
Fix docstring for dbt docs/source to not include misleading flags
2020-02-07 12:58:32 -07:00
Jacob Beck
bf3f5e1b92
Attach cli vars to configs when they come in via the RPC server
2020-02-07 12:39:21 -07:00
Jacob Beck
20496651c5
depends_on macros
...
Clean up macro generation a bit
2020-02-07 12:11:08 -07:00
Jacob Beck
9df123a180
Merge pull request #2085 from fishtown-analytics/feature/contexts-redux
...
Feature: contexts cleanup
2020-02-07 12:09:23 -07:00
Brayden Connole
e7b597787d
Fix doc string duplicate issue and add test
2020-02-06 15:42:07 -07:00
Jacob Beck
048b96b540
Render all description fields, regardless of docrefs
...
remove the whole idea of docrefs
2020-02-06 12:03:31 -07:00
Jacob Beck
be29156f6f
add macro argument definitions
2020-02-06 12:03:31 -07:00
Jacob Beck
140cfd70ce
Merge pull request #2093 from fishtown-analytics/fix/generate-no-compile
...
fix --no-compile flag for docs generate
2020-02-06 10:51:57 -07:00
Jacob Beck
2e5fdbf5ce
PR feedback
...
Fix comments
Implement a TODO around duplicate macro names
- added unit tests for it
Implement a TODO around ref resolution
2020-02-06 08:59:09 -07:00
Jacob Beck
ee710e3922
add some tests
...
Make slight tweaks to project initialization to support tests
2020-02-06 08:40:58 -07:00
Jacob Beck
b8febddad5
dbt Contexts
...
Rewrite/reorganize contexts
- now they are objects that have a to_dict()
- special decorators on properties/methods to indicate membership
- reorganize contexts/ to remove many many import cycles
Context behavior changes:
- 'dbt_version' is alwways available
- 'target' is available as long as the profile has been parsed
- 'project_name' is available in: query headers, hooks, models, and macro execution
Profiles/projects now render at load time
- reading a profile or project file requires a ConfigRenderer
- projects get an extra-fiddly special load for use during initial parsing
- it returns the profile name and a function that, given a renderer, can return the rest of the Project
- idea is: use the profile name to load the Profile, use that to build a ConfigRenderer that has a TargetContext, render the project with that
- profiles.yml is rendered with the 'base' context
- dbt_project.yml/schema.yml/packages.yml are rendered with the 'target' context: 'base' context + 'target'
- docs are rendered with the docs context: 'target' context + the 'doc' function
- query headers are rendered with the query header context: 'target' context + macros + 'project_name'
- executed macros/models should have the same context as previously (query headers + adapter/model/etc related functions)
Moved actual ref/source searching into the manifest
Moved the rest of the parse utils into parser/manifest.py
Made the ref/source resolvers defined in the provider context a bit more sane/well-defined
Picked consistent-but-not-great names for all the various context generation functions
Moved write_node into ParsedNode
2020-02-06 08:40:57 -07:00
Jacob Beck
da74681de2
PR feedback
2020-02-06 07:54:24 -07:00
Jacob Beck
c8605647f1
fix --no-compile flag for docs generate
2020-02-06 07:53:40 -07:00
Jacob Beck
032d77dd15
Merge pull request #2094 from fishtown-analytics/fix/debug-from-subdir
...
allow "dbt debug" from subdirectories (#2086 )
2020-02-06 07:37:33 -07:00
Jacob Beck
0df49c59c0
Merge pull request #2037 from fishtown-analytics/feature/faster-snowflake-catalogs
...
Feature: faster snowflake catalogs (#2009 )
2020-02-06 07:36:58 -07:00
sonac
d354aa6f09
removing trailing space
2020-02-06 10:01:51 +01:00
Matt Ball
551bf42c66
Update DEBUG_LOG_FORMAT to correctly zero-pad the microsecond portion of the timestamp
2020-02-05 16:42:53 -07:00
Jacob Beck
b691a73dba
Bump version: 0.15.2 → 0.16.0a2
2020-02-04 19:28:34 -07:00
Jacob Beck
26720997ea
Merge branch 'dev/0.15.2' into dev/barbara-gittings
2020-02-04 19:28:15 -07:00
Jacob Beck
431ce50964
allow "dbt debug" from subdirectories
...
When dbt_project.yml is missing, fail more gracefully
When dbt_project.yml is missing and no profile is provided, validate all profiles:
- read everything in profiles.yml
- validate the default target for each profile
- print no connection info
- mark profiles.yml as valid
2020-02-04 13:19:52 -07:00
Jacob Beck
04bc2a800a
PR feedback
...
Added a custom table merge implementation that tracks if a row is all null and merges those as "any type".
- added unit tests for that!
Removed some schema casing things
fixed pluralization (it was reversed)
2020-02-04 07:48:00 -07:00
Jacob Beck
ccab27a1e2
Merge pull request #2089 from fishtown-analytics/feature/batched-tracking
...
batch anonymous usage statistics calls
2020-02-04 06:45:25 -07:00
sonac
fa40c41124
splitting long line
2020-02-04 10:08:03 +01:00
Jacob Beck
1881c0b932
batching
2020-02-03 08:56:30 -07:00
Connor McArthur
0d44dbf078
Bump version: 0.15.2a1 → 0.15.2
2020-02-02 14:16:13 -05:00
Connor McArthur
c5a6634d59
changelog
2020-02-02 14:16:07 -05:00
Connor McArthur
81dee7b857
Merge branch '0.15.latest' of github.com:fishtown-analytics/dbt into dev/0.15.2
2020-02-02 14:14:15 -05:00
Drew Banin
b1f88cb1a3
Merge pull request #2087 from fishtown-analytics/docs/0.15.2
...
Update CHANGELOG.md
2020-01-31 17:13:34 -05:00
Drew Banin
f2ba4b03a2
Update CHANGELOG.md
2020-01-31 17:11:48 -05:00
Jacob Beck
c1af3abbdc
PR feedback w/ improved catalog results behavior
2020-01-31 11:52:43 -07:00
Jacob Beck
0bf6ecafa0
migrate other adapters to use the snowflake technique
2020-01-31 10:56:29 -07:00
Jacob Beck
0658a420ee
Attempt getting snowflake to go faster and use less ram
...
- Thread information schema queries, one per db
- Pass schema list into catalog queries and filter on that in SQL
- break the existing interface for get_catalog (sorry, not sorry)
2020-01-31 10:53:15 -07:00
sonac
5823683265
injecting only message from validation error
2020-01-31 17:03:44 +01:00
Jacob Beck
5b65591cc2
Merge pull request #2068 from fishtown-analytics/feature/macro-docs
...
Add documentation for macros/analyses (#1041 )
2020-01-31 07:28:06 -07:00
Andrii
7fae368c0d
Apply suggestions from code review
...
adding actual error text into wrapper
Co-Authored-By: Drew Banin <drew@fishtownanalytics.com >
2020-01-31 13:36:07 +02:00
Jacob Beck
04ebdbb00b
PR feedback
2020-01-30 21:18:46 -07:00
Jacob Beck
b030b4eba5
Add documentation for macros/analyses
...
Refactored parsing a bit to support the idea of:
- macro patches (no columns)
- node patches that don't get tests (analyses)
2020-01-30 21:12:34 -07:00
Jacob Beck
b01226fe62
Merge pull request #2035 from fishtown-analytics/fix/manifest-search
...
Fix manifest search (#2032 )
2020-01-30 21:10:47 -07:00
Jacob Beck
e080bfc79a
Merge pull request #2069 from fishtown-analytics/feature/snowflake-oauth
...
add oauth authentication to snowflake adapter (#2050 )
2020-01-30 15:51:48 -07:00
sonac
9eaf2438dd
making error msg for malformed packages.yml better
2020-01-30 10:56:31 +01:00
Drew Banin
bd5e6bd1e6
Merge pull request #2076 from markberger/setuptools-check
...
add setuptools version check for overall setup.py
2020-01-29 15:32:22 -05:00
Jacob Beck
62755fe5b1
More macro search changes
...
Unify the lookup for the various macros, all of them are done in the manifest now:
- dbt's built-in package + all plugins are part of 'core'
- finding macros has the concept of an optional package name
- materializations are a special case of finding macros
- the `generate_*_name` macros are another special case (non-core dependency packages are ignored)
- generating the macro execution context now uses the concept of an optional search_package_name
- macro execution now binds these two together
Unify the lookup for models/docs/sources
- added a search_name property to those three types
- create a 'Searchable' protocol, have the node search code accept that
- simplify matching logic/push it into the correct types accordingly
Rename get_materialization_macro to find_materialization_macro_by_name (consistency)
context namespacing behavior:
- dbt run-operation now passes the named package along to macro execution if a package name is specified
- so `dbt run-operation dependency.ad` runs with the dependency namespace as local, overriding globals
- but `dbt run-operation my_macro` runs in the current package namespace as local, even if it ultimately runs the dependency's my_macro()
- the current package namespace is always treated as "global", overriding core macros
Tons of tests
2020-01-29 13:29:43 -07:00
Jacob Beck
e570d22f40
Merge pull request #2010 from fishtown-analytics/fix/mypy-cleanups
...
Fix some mypy checking
2020-01-29 13:19:49 -07:00
Jacob Beck
3e48dc3b4d
PR feedback
2020-01-29 10:15:48 -07:00
Mark Berger
9b06679d56
add setuptools version check for overall setup.py
2020-01-28 16:51:26 -08:00
Jacob Beck
9cc7a7a87f
Fix mypy checking
...
Make mypy check our nested namespace packages by putting dbt in the mypy_path.
Fix a number of exposed mypy/type checker complaints. The checker mostly
passes now even if you add `--check-untyped-defs`, though there are a couple lingering issues so I'll leave that out of CI
Change the return type of RunOperation a bit - adds a couple fields to appease mypy
Also, bump the mypy version (it catches a few more issues).
2020-01-27 12:48:10 -07:00
Jacob Beck
4e23e7dbd1
Merge pull request #2039 from fishtown-analytics/feature/source-tags
...
Tags for sources and columns (#1906 , #1586 )
2020-01-27 11:26:25 -07:00
Jacob Beck
d9ec8125c9
Merge pull request #2071 from fishtown-analytics/fix/avoid-disabled-models-crash
...
Fix crash on disabled models (#2070 )
2020-01-27 11:26:20 -07:00
Drew Banin
73c418cf57
Merge pull request #2060 from emilieschario/dev/0.15.1
...
Add logging for a file path that can't be cleaned
2020-01-27 12:56:36 -05:00
Jacob Beck
8b722c7951
allow single name tags in test configs
2020-01-27 10:38:26 -07:00
Drew Banin
fdfcd4c651
Merge pull request #2062 from fishtown-analytics/fix/overwrite-clobber-config-fields
...
Overwrite clobber config fields
2020-01-27 12:23:37 -05:00
Jacob Beck
4357c3c74e
fix partial parsing to avoid a KeyError, tests
2020-01-27 10:19:59 -07:00
Jacob Beck
5c80d6ab4e
Add a tags parameter to tests, like severity
...
- edited a test to exercise it
2020-01-27 09:30:58 -07:00
emilielimaburke
199b1ed1dc
fix protected_abs_paths
2020-01-25 18:00:45 -05:00
Jacob Beck
b9f8dedc8b
Merge branch 'dev/0.15.2' into dev/barbara-gittings
2020-01-24 13:56:59 -07:00
Jacob Beck
96a9c6cb92
Merge pull request #2067 from fishtown-analytics/fix/docs-missing-tag-hang
...
fix an RPC hang when enddocs tags were missing (#2066 )
2020-01-24 13:38:34 -07:00
Jacob Beck
2ad5122ef4
Add tags for sources
...
Allow column-level tags, which are inherited by their tests
2020-01-24 13:33:55 -07:00
Drew Banin
1f749805a5
( #2049 ) Fix for clobber config fields which were errantly extended
2020-01-24 15:22:44 -05:00
Jacob Beck
de5ff68943
PR feedback: lines not chars
2020-01-24 13:08:53 -07:00
Jacob Beck
e7d2221d08
add oauth authentication to snowflake adapter
2020-01-24 10:26:16 -07:00
Jacob Beck
0b8f3bc4c5
Merge pull request #2065 from fishtown-analytics/fix/ignore-alternative-resource-types
...
ignore alternative resource types when reading partial parse cache (#2064 )
2020-01-24 10:20:59 -07:00
Jacob Beck
9d5488e361
Merge pull request #2051 from fishtown-analytics/feature/docs-for-everyone
...
Make dbt support documentation for snapshots and seeds (and analyses) (#1974 )
2020-01-24 09:45:53 -07:00
Jacob Beck
80574a5a6a
Merge pull request #2046 from fishtown-analytics/feature/is-number
...
Add Column.is_number/is_float (#1969 )
2020-01-24 09:43:21 -07:00
Jacob Beck
affdbe719a
PR feedback
2020-01-24 08:27:19 -07:00
Jacob Beck
ba3e14cefd
fix an RPC hang when enddocs tags were missing, add tests
2020-01-24 08:21:58 -07:00
emilielimaburke
105dc001e5
put on two lines
2020-01-23 20:24:01 -05:00
Jacob Beck
8bdaa69a2c
flake8
2020-01-23 11:08:21 -07:00
Jacob Beck
760d46af5c
when going through the partial parsing, ignore cached nodes that do not match the current parser resource type
2020-01-22 12:51:35 -07:00
Jacob Beck
21c3f41814
fix dedent/formatting with "Deprecation Warning" prefix
2020-01-22 11:42:48 -07:00
Jacob Beck
5c42f1af14
Merge pull request #2058 from fishtown-analytics/feature/expand-schema-search
...
expand search for docs and schema.yml (#1832 )
2020-01-22 11:25:01 -07:00
Jacob Beck
7a89ef2465
PR feedback
2020-01-22 10:20:26 -07:00
Drew Banin
38244bfdb3
Merge pull request #2026 from NiallRees/fix/disabled_tests_in_stats
...
Exclude tests for disabled models in compile stats
2020-01-20 22:43:23 -05:00
Emilie Lima Schario
e7759b4bae
Update error message with error warning
2020-01-20 19:56:59 -05:00
Jacob Beck
66a4f76c74
Merge pull request #2056 from fishtown-analytics/fix/partial-parsing-disabled-models
...
Fix partial parsing with disabled models (#2055 )
2020-01-18 21:37:04 -07:00
emilielimaburke
80c9e623ef
add logging for a file path that can't be cleaned
2020-01-18 15:28:05 -05:00
Drew Banin
722d87ca5c
Merge pull request #2057 from aescay/feature/add_target_flag_alias
...
Add -t shorthand for --targets
2020-01-17 14:57:31 -05:00
Jacob Beck
f0221b1fdb
expand search for docs and schema.yml
...
update tests
appease mypy
2020-01-17 11:34:03 -07:00
aescay
51e65a35a6
add -t shorthand for --targets
2020-01-17 11:49:18 -05:00
Jacob Beck
a8b93f6d48
fix an issue where partial parsing assumed model ids could not overlap between disabled and enabled
2020-01-17 09:32:05 -07:00
Jacob Beck
9b7de69f08
Bump version: 0.15.1 → 0.15.2a1
2020-01-17 09:31:00 -07:00
NiallRees
554b32efb5
Split expression onto two lines
2020-01-16 22:32:42 +00:00
Jacob Beck
c4a5eb9803
PR feedback
...
add support for newlines in output
line-wrap deprecations
2020-01-16 09:19:10 -07:00
Jacob Beck
f92f389612
Make dbt support documentation for snapshots and seeds (and analyses)
...
Add a deprecation warning for snapshots/seeds documented under "models"
Add a bunch of tests
2020-01-16 08:02:09 -07:00
Jacob Beck
1021637283
Merge pull request #2045 from fishtown-analytics/feature/detect-duplicate-macros
...
Detect duplicate macros (#1891 )
2020-01-16 08:01:22 -07:00
Jacob Beck
d85a54a01c
PR feedback: improve error message formatting
2020-01-16 07:32:34 -07:00
Connor McArthur
77dceab7be
Merge pull request #2053 from fishtown-analytics/dev/0.15.1
...
dbt 0.15.1
2020-01-16 09:27:54 -05:00
Jacob Beck
01e97ff285
Merge pull request #2042 from fishtown-analytics/feature/select-on-seeds
...
support --select on dbt seed (#1711 )
2020-01-16 07:19:49 -07:00
Connor McArthur
f6f0f5ce1e
Bump version: 0.15.1rc2 → 0.15.1
2020-01-16 08:46:19 -05:00
Connor McArthur
75a5c09981
Merge branch 'dev/0.15.1' of github.com:fishtown-analytics/dbt into dev/0.15.1
2020-01-16 08:44:51 -05:00
Connor McArthur
16674275ed
add changelog entry for source snapshot-freshness command
2020-01-16 08:44:49 -05:00
Connor McArthur
a0524af242
Merge pull request #2041 from fishtown-analytics/fix/source-freshness-rpc
...
Add source snapshot-freshness to dbt rpc (#2040 )
2020-01-16 08:43:23 -05:00
Jacob Beck
54b64f8922
add "is_number" and "is_float" Column methods
...
Split out snowflake column type
added column type test integration tests
2020-01-13 16:32:50 -07:00
Jacob Beck
706cea8996
detect duplicate macro names in the same project, and raise an exception about it
2020-01-13 12:19:28 -07:00
Jacob Beck
1a72660a3f
Merge pull request #1989 from franloza/fix/project-dir-debug
...
Fix project dir argument when running debug (#1733 )
2020-01-10 15:31:59 -07:00
Jacob Beck
efe75d9b51
Add support for --select and --exclude for seeds
2020-01-10 14:51:19 -07:00
Jacob Beck
2ac24991c1
Add source snapshot-freshness to dbt rpc
2020-01-10 14:16:40 -07:00
Jacob Beck
b11aab5369
Merge pull request #2036 from fishtown-analytics/feature/toyaml-fromyaml
...
add toyaml/fromyaml (#1911 )
2020-01-10 12:20:34 -07:00
Jacob Beck
a348c3f7c1
Merge branch 'dev/0.15.1' into dev/barbara-gittings
2020-01-08 13:31:52 -07:00
Jacob Beck
78a731b9e2
add toyaml/fromyaml
2020-01-08 13:23:17 -07:00
Fran Lozano
94ede83462
Remove unnecessary call to use_default_project call from test
2020-01-08 00:01:26 +01:00
Fran Lozano
1085fb8cf4
Split line too long
2020-01-07 23:46:05 +01:00
Drew Banin
0a5b436da1
Merge pull request #2015 from tayloramurphy/add-data-dictionary
...
Handle "meta:" key in schema/sources.yml
2020-01-07 10:49:29 -05:00
Drew Banin
7d3591f5c1
Merge pull request #2012 from fishtown-analytics/fix/pedantic-pluralize
...
Fix: pedantic pluralization
2020-01-06 17:23:12 -05:00
Drew Banin
4ed1986c26
Merge pull request #2028 from alanmcruickshank/ref_override
...
Allow ref override
2020-01-06 17:19:33 -05:00
Jacob Beck
33b4e72401
Merge pull request #2022 from fishtown-analytics/fix/better-plugin-import-errors
...
improve errors on importing plugins (#2006 )
2020-01-06 15:11:29 -07:00
Jacob Beck
c36a463f44
Merge pull request #2024 from fishtown-analytics/fix/ephemeral-cancelled
...
Fix cancelled ephemeral models (#1993 )
2020-01-06 15:11:09 -07:00
Jacob Beck
f85015365f
Merge pull request #2025 from fishtown-analytics/fix/parse-refs-in-hooks
...
Parse model hooks for refs (#1957 )
2020-01-06 15:05:15 -07:00
Jacob Beck
b9a47cafb2
Merge pull request #2031 from fishtown-analytics/feature/improve-on-run-end
...
improve on run end (#1924 )
2020-01-06 15:05:05 -07:00
Jacob Beck
22ceecaa87
Merge pull request #2023 from fishtown-analytics/fix/unit-test-concurrency
...
make unit tests use a one-per-test temp directory
2020-01-06 15:04:01 -07:00
Alan Cruickshank
37e373a68f
Allow ref override + tests
2020-01-06 18:15:09 +00:00
Jacob Beck
ff9013d977
add db, schema pairs to the context
...
Update tests to use that information
2020-01-06 10:47:37 -07:00
Jacob Beck
39f92e527c
Merge pull request #2030 from fishtown-analytics/fix/detect-deps-same-name-as-root
...
detect deps with the same name as the root
2020-01-05 11:06:25 -07:00
Jacob Beck
f1ef68c166
fix the unit tests
2020-01-05 10:32:38 -07:00
Jacob Beck
260752f501
when the root project name is the same as a dependency project name, raise an error
2020-01-03 19:13:51 -07:00
Connor McArthur
01c23c3414
Bump version: 0.15.1rc1 → 0.15.1rc2
2019-12-30 13:44:38 -05:00
Taylor A. Murphy
dea5f98c4e
Additional meta references
2019-12-30 09:12:38 -06:00
Niall Woodward
38d51cd97c
Exclude tests for disabled models in compile stats
2019-12-28 12:06:12 +00:00
Jacob Beck
22db36c4a6
Parse model hooks for refs
...
Add a parse pass for model hooks, parse them like models
Attach the hook refs as if they were on the model
Add a test exercising refs
2019-12-24 19:06:42 -07:00
Connor McArthur
71a293ce46
Merge pull request #2021 from fishtown-analytics/0.15.1-changelog-rc-2
...
Update CHANGELOG.md
2019-12-24 14:25:25 -05:00
Jacob Beck
75356cd9ce
suppress cancel logs on ephemeral nodes
...
During ctrl+c handling, check the name of the canceled connection
if it's the name of an ephemeral node, suppress the cancel message
2019-12-23 14:33:41 -07:00
Jacob Beck
c7e2d2270d
make unit tests use a one-per-test temp directory - fix concurrent rm/create conflict
2019-12-23 13:04:49 -07:00
Jacob Beck
df05037841
improve errors on import to be more specifically correct
...
When the error is about not being able to import the plugin, indicate that
Otherwise log the original error stack trace at debug and re-raise it
2019-12-23 11:28:40 -07:00
Jacob Beck
1235b3f468
Merge branch 'dev/0.15.1' into dev/barbara-gittings
2019-12-23 09:48:11 -07:00
Fran Lozano
99e778b9b7
Update order of imports in test_debug
2019-12-21 14:19:27 +01:00
Fran Lozano
c1b3690671
Use --project-dir outside current dir in debug when it exists #1733
2019-12-21 14:06:01 +01:00
Drew Banin
df1384bf31
Update CHANGELOG.md
2019-12-20 16:38:28 -05:00
Drew Banin
5a4ddd685d
Merge pull request #1967 from kconvey/feature/bq_sql_header
...
Add support for sql as a header to create or replace
2019-12-20 16:35:37 -05:00
Jacob Beck
8cd8705d67
Merge pull request #2018 from fishtown-analytics/fix/cleanup-retries
...
clean up some retry logic (#2016 )
2019-12-19 15:02:28 -07:00
Taylor A. Murphy
4927674306
Alter formatting for another commit
2019-12-19 13:55:07 -06:00
Taylor A. Murphy
eb464752fb
Bane of my existence
2019-12-19 13:42:17 -06:00
Taylor A. Murphy
210092eeda
Update docs blocks integration test
2019-12-19 13:20:24 -06:00
Taylor A. Murphy
9f7ecd28f9
Update docs generate integration test
2019-12-19 13:18:41 -06:00
Taylor A. Murphy
a05b013ecc
Fix a few more tests
2019-12-19 12:32:00 -06:00
Taylor A. Murphy
bc49fc2e10
Remove straggling comma
2019-12-19 12:12:21 -06:00
Taylor A. Murphy
344af97656
Forgot some commas
2019-12-19 12:01:53 -06:00
Taylor A. Murphy
ac650c1034
Update ColumnInfo tests
2019-12-19 11:32:17 -06:00
Taylor A. Murphy
b411630ad4
More test updates
2019-12-19 11:24:19 -06:00
Jacob Beck
86b917fd33
fixes
2019-12-19 10:22:10 -07:00
Taylor A. Murphy
c1815d6750
Update tests
2019-12-19 11:12:06 -06:00
Kurt Convey
e6ab64ee45
Fix expected number of test models
2019-12-19 09:39:50 -07:00
Taylor A. Murphy
f7d39ae7b3
Switch to 'meta' from 'data'
2019-12-19 10:25:24 -06:00
Kurt Convey
98c6eace0c
materialize test model as table
2019-12-18 14:26:37 -07:00
Connor McArthur
6be4ee00aa
Bump version: 0.15.0 → 0.15.1rc1
2019-12-18 15:37:30 -05:00
Connor McArthur
88b35f7ee1
Merge pull request #2014 from fishtown-analytics/changelog/0.15.1
...
Update CHANGELOG.md for v0.15.1
2019-12-18 15:36:25 -05:00
Drew Banin
27212a16dd
Update CHANGELOG.md
2019-12-18 15:33:20 -05:00
Jacob Beck
74672c8fad
Merge branch 'dev/0.15.1' into dev/barbara-gittings
2019-12-18 12:58:21 -07:00
Taylor A. Murphy
d4108ca840
Add data from sources to manifest.json
2019-12-18 12:39:41 -06:00
Taylor A. Murphy
499aa57ac2
Handle "data:" key in schema/sources.yml
2019-12-18 12:04:49 -06:00
Drew Banin
0b05bf0fa6
Update CHANGELOG.md
...
Co-Authored-By: Jacob Beck <beckjake@users.noreply.github.com >
2019-12-18 11:59:59 -05:00
Jeremy Cohen
d98613db13
Mutatis mutandis
2019-12-18 10:06:44 -05:00
Drew Banin
c4baf72fd4
Update CHANGELOG.md
2019-12-18 09:55:15 -05:00
Jeremy Cohen
d725c7c5e6
One analysis, two analyses
2019-12-17 21:35:23 -05:00
Kurt Convey
0dfb5d4441
Fix merge conflicts and integration test
2019-12-17 10:42:50 -07:00
Kurt Convey
d9ee6ea917
Add semicolon after UDF
2019-12-16 11:02:44 -07:00
Jacob Beck
0fe5fb628a
Merge pull request #2007 from fishtown-analytics/fix/bigquery-upper-bounds
...
put an upper bound on bigquery
2019-12-13 14:31:40 -07:00
Drew Banin
fea06c6fdb
Merge pull request #1964 from kconvey/feature/bq_labels
...
BigQuery Labels
2019-12-13 15:58:45 -05:00
Jacob Beck
02c2926ea4
Merge pull request #2005 from fishtown-analytics/fix/bigquery-catalog-info-schema-queries
...
Fix bigquery catalog queries with nonexistent schemas (#1984 )
2019-12-13 13:58:31 -07:00
Jacob Beck
b151e2a15a
Merge pull request #2001 from fishtown-analytics/fix/snapshot-check-all-added-column
...
Fix snapshot check all with an added column (#1797 )
2019-12-13 13:49:25 -07:00
Jacob Beck
a78b4fda8f
put an upper bound on bigquery, add six
2019-12-13 11:42:33 -07:00
Jacob Beck
a702f58ed7
Merge pull request #1994 from fishtown-analytics/fix/overlapping-valid-timestamps
...
Avoid overlapping validity timestamps (#1736 )
2019-12-13 10:22:48 -07:00
Jacob Beck
8311285adb
In bigquery, filter out missing schemas before querying the catalog
2019-12-13 09:17:22 -07:00
Jacob Beck
62e22c27d5
Merge pull request #1920 from fishtown-analytics/feature/tighten-agate-values
...
Agate fixes (#999 ) (#1639 )
2019-12-13 08:42:22 -07:00
Jacob Beck
89dd04d06e
Merge pull request #2002 from fishtown-analytics/fix/rpc-thread-names
...
call pull_information() before we enqueue the log record (#1905 )
2019-12-13 08:38:23 -07:00
Jacob Beck
9222f803f4
Merge pull request #1963 from kconvey/feature/retries
...
Implement retries in BQ adapter
2019-12-12 11:25:34 -07:00
Jacob Beck
2456b6dc8b
Merge pull request #2003 from fishtown-analytics/fix/postgres-redshift-wildcard-pg
...
escape the wildcard underscore in postgres "like" queries (#1960 )
2019-12-12 10:33:44 -07:00
Jacob Beck
91b97641a2
tests
2019-12-12 09:30:15 -07:00
Jacob Beck
123c3fa78b
escape the wildcard underscore in postgres "like" queries
2019-12-12 08:15:43 -07:00
Jacob Beck
ab4925f59f
this is ok now
2019-12-11 13:30:04 -07:00
Kurt Convey
0356a74d4c
clean up merge conflict
2019-12-11 13:18:51 -07:00
Kurt Convey
37a1288ab0
Merge branch 'feature/retries' of github.com:kconvey/dbt into feature/retries
2019-12-11 12:59:04 -07:00
Kurt Convey
43959dc4d0
Return function results
2019-12-11 12:57:32 -07:00
Jacob Beck
11e5e1b0f2
call pull_information() before we enqueue the log record
2019-12-11 11:59:59 -07:00
Jacob Beck
9589e7aaa1
Fix snapshot check all strategy with added column
2019-12-11 11:21:29 -07:00
Kurt
c5c7932ee7
Update plugins/bigquery/dbt/adapters/bigquery/connections.py
...
Co-Authored-By: Jacob Beck <beckjake@users.noreply.github.com >
2019-12-11 12:47:41 -05:00
Kurt Convey
5d181c3ef2
Clean up tests
2019-12-11 08:50:21 -07:00
Kurt
ce4c58a1f0
Update plugins/bigquery/dbt/adapters/bigquery/connections.py
...
Co-Authored-By: Jacob Beck <beckjake@users.noreply.github.com >
2019-12-10 12:56:33 -05:00
Kurt
e6b4a12c08
Update plugins/bigquery/dbt/adapters/bigquery/connections.py
...
Co-Authored-By: Jacob Beck <beckjake@users.noreply.github.com >
2019-12-10 12:56:24 -05:00
Kurt
7c8a21ddcf
Update plugins/bigquery/dbt/adapters/bigquery/connections.py
...
Co-Authored-By: Jacob Beck <beckjake@users.noreply.github.com >
2019-12-10 12:55:56 -05:00
Jacob Beck
40839c79fc
Merge pull request #1996 from Fokko/remove-duplicates
...
Remove the duplicate get_context_modules
2019-12-10 09:40:01 -07:00
Fokko Driesprong
284f5a4968
Add mypy ignore
2019-12-10 15:53:49 +01:00
Fokko Driesprong
6241ff4bef
Whoops, missed that one
2019-12-10 15:43:22 +01:00
Fokko Driesprong
cf41203504
Remove the duplicate get_context_modules
2019-12-10 15:38:22 +01:00
Kurt Convey
460d73f55e
Fix formatting
2019-12-09 16:47:50 -07:00
Kurt Convey
09403094cd
Still get table ref
2019-12-09 15:10:38 -07:00
Kurt
40c9328e84
Apply suggestions from code review
...
Clean up retries unit test's connection manager mocking
Co-Authored-By: Jacob Beck <beckjake@users.noreply.github.com >
2019-12-09 17:09:28 -05:00
Jacob Beck
382a993e68
make the test even more strict
2019-12-09 15:05:29 -07:00
Kurt Convey
0f05404f72
Add sql_header common default
2019-12-09 14:45:02 -07:00
Jacob Beck
33836ea5f8
Avoid overlapping validity timestamps, tests
2019-12-09 14:21:31 -07:00
Jacob Beck
cc491904bc
Merge pull request #1992 from fishtown-analytics/feature/lazy-load-connections
...
lazy-load connections (#1584 )
2019-12-09 12:02:48 -07:00
Jacob Beck
be36c5d974
lazy-load connections
2019-12-09 11:57:43 -07:00
Jacob Beck
2dd604b039
Merge pull request #1976 from fishtown-analytics/fix/override-materializations
...
add a formal search order for materializations, tests (#1962 )
2019-12-09 11:43:01 -07:00
Fran Lozano
58a371f9d9
Fix #1733
2019-12-08 01:09:28 +01:00
Jacob Beck
564cc22d95
Merge pull request #1988 from Fokko/fd-use-generated-at
...
Use generated_at instead of datetime.utcnow()
2019-12-06 09:45:36 -07:00
Jacob Beck
1da96b56e5
Merge pull request #1982 from Fokko/fd-add-annotations
...
Add annotations to the code
2019-12-06 09:10:01 -07:00
Fokko Driesprong
5a929b095c
Limit the line length
2019-12-06 16:00:53 +01:00
Fokko Driesprong
3bc4834c82
Use generated_at instead of datetime.utcnow()
2019-12-06 09:15:49 +01:00
Fokko Driesprong
43213cddfd
Merge branch 'dev/0.15.1' into fd-add-annotations
2019-12-06 08:49:45 +01:00
Fokko Driesprong
9975cd5f27
Fix code style voilations
2019-12-06 08:36:32 +01:00
Jacob Beck
65b4c18ab6
Merge pull request #1977 from fishtown-analytics/feature/emit-warning-context
...
add exceptions.warn to the dbt context (#1970 )
2019-12-05 16:45:48 -07:00
Jacob Beck
d14d250e0a
Merge pull request #1983 from fishtown-analytics/fix/restore-drop-schema
...
Restore drop_schema to adapter in jinja context (#1980 )
2019-12-05 16:45:28 -07:00
Jacob Beck
a993d9c355
Merge pull request #1978 from fishtown-analytics/feature/detect-downlevel-setuptools
...
fail more gracefully when setuptools is downlevel (#1975 )
2019-12-05 16:44:52 -07:00
Jacob Beck
14c4bc7967
rename emit_warning -> warn
2019-12-05 14:26:27 -07:00
Jacob Beck
143402dace
do not commit after dropping the schema
2019-12-05 14:23:52 -07:00
Jacob Beck
3dff3e71bc
PR feedback
2019-12-05 14:20:24 -07:00
Jacob Beck
549168b835
Merge pull request #1979 from fishtown-analytics/feature/configurable-test-warehouses
...
make tests honor environment variables (#1939 )
2019-12-05 14:18:00 -07:00
Fokko Driesprong
d4c1dbb133
Remove BaseRelation
2019-12-05 20:53:10 +01:00
Fokko Driesprong
ca454360d8
Process comments
2019-12-05 20:34:03 +01:00
Fokko Driesprong
2f17424b89
Add annotations to the code
2019-12-05 19:24:56 +01:00
Jacob Beck
b80fd81e65
Restore drop_schema to contexts
...
Made a few related fixes to caching behavior w.r.t schemas
Added a test
2019-12-05 11:15:49 -07:00
Jacob Beck
5797be9b14
make tests honor environment variables
2019-12-04 14:49:36 -07:00
Jacob Beck
a771c637d7
fail more gracefully when setuptools is missing
2019-12-04 14:36:15 -07:00
Jacob Beck
dd5e47f2e0
add exceptions.emit_warning to the dbt context
2019-12-04 14:10:17 -07:00
Jacob Beck
1b0d635152
add a formal search order for materializations, tests
2019-12-04 10:45:02 -07:00
Drew Banin
ace777e495
Merge pull request #1973 from fishtown-analytics/drew-update-contributing-guidelines
...
update contributing guidelines and PR template
2019-12-04 12:09:45 -05:00
Drew Banin
e882ed39dc
Update CONTRIBUTING.md
...
Co-Authored-By: Connor McArthur <connor@fishtownanalytics.com >
2019-12-04 12:09:31 -05:00
Kurt Convey
7c6f878c69
Update bq integration test's expected number of models
2019-12-04 09:50:32 -07:00
Kurt Convey
fc94a5e19d
Add snowflake support and test UDF
2019-12-04 09:21:03 -07:00
Drew Banin
7538f1eb9c
Update CONTRIBUTING.md
2019-12-03 21:59:17 -05:00
Drew Banin
43028fe89d
Update pull_request_template.md
2019-12-03 20:15:41 -05:00
Drew Banin
5f9b8b92ba
Create pull_request_template.md
2019-12-03 20:13:55 -05:00
Drew Banin
4c865ac793
Update CONTRIBUTING.md
2019-12-03 20:01:00 -05:00
Kurt Convey
7a51ef664a
Another newline
2019-12-03 16:04:58 -07:00
Kurt Convey
9e9f0307df
newline and better test
2019-12-03 16:04:19 -07:00
Kurt Convey
5eafbb794a
Make sql_header universal accross adapters
2019-12-03 16:01:52 -07:00
Kurt Convey
b5483754d0
Merge branch 'feature/retries' of https://github.com/kconvey/dbt into feature/retries
2019-12-03 15:21:39 -07:00
Kurt Convey
e03fd44d6d
Use client.create_dataset and client.delete_dataset
2019-12-03 15:19:58 -07:00
Kurt
9b9c1db05e
Update plugins/bigquery/dbt/adapters/bigquery/connections.py
...
Co-Authored-By: Drew Banin <drew@fishtownanalytics.com >
2019-12-03 15:02:05 -05:00
Kurt Convey
63d14f3a0c
Fix integration test and enable labels for views as well
2019-12-03 12:57:32 -07:00
Kurt Convey
bef02456eb
newline
2019-12-02 15:44:56 -07:00
Kurt Convey
a4efd223a4
Add integration test and implement suggestions
2019-12-02 15:42:45 -07:00
Kurt Convey
5058049096
Add support for sql as a header to create or replace
2019-12-02 10:45:26 -07:00
Kurt Convey
c0e854007e
Remove fake credentials in unit test
2019-11-27 17:22:48 -07:00
Kurt Convey
118344ba62
Mock
2019-11-27 14:29:47 -07:00
Kurt Convey
33e75f8fe1
Mock
2019-11-27 14:17:21 -07:00
Kurt Convey
03472381a0
Mock
2019-11-27 13:56:14 -07:00
Kurt Convey
3aabe2da1a
Add query_comment
2019-11-27 13:47:53 -07:00
Kurt Convey
ad0bd8763a
fix
2019-11-27 13:27:34 -07:00
Kurt Convey
3b696eedb2
fix test better
2019-11-27 13:13:34 -07:00
Kurt Convey
b2c1727b67
Fix test
2019-11-27 12:52:49 -07:00
Kurt Convey
b602c9cdf6
Fix
2019-11-27 12:05:09 -07:00
Kurt Convey
86f0609931
blank line
2019-11-27 08:36:07 -07:00
Kurt Convey
3b456593a6
fix imports
2019-11-27 08:33:19 -07:00
Kurt Convey
a3e8eabc66
cleanup
2019-11-27 08:28:02 -07:00
Kurt Convey
df2c498e9a
Implement bq labels as a configurable option
2019-11-27 08:22:37 -07:00
Kurt Convey
21da0ed5b3
Implement retries in BQ adapter
2019-11-27 06:03:44 -07:00
Connor McArthur
e51c942e91
Bump version: 0.15.0rc2 → 0.15.0
2019-11-25 13:38:30 -05:00
Connor McArthur
9dec850072
set release date for 0.15.0
2019-11-25 13:38:06 -05:00
Drew Banin
7814ea05be
Merge pull request #1953 from fishtown-analytics/fix/0.15.0-changelog-docs
...
Update CHANGELOG.md
2019-11-25 10:22:33 -05:00
Drew Banin
7f9750dd4f
Update CHANGELOG.md
2019-11-25 10:20:42 -05:00
Jacob Beck
5b1e4f56a0
Merge pull request #1951 from fishtown-analytics/fix/change-psycopg-name
...
Use 'psycopg2-binary' package by default on all platforms
2019-11-25 08:15:54 -07:00
Drew Banin
9a9f705c42
Merge pull request #1954 from fishtown-analytics/fix/lma-contributing
...
Update CONTRIBUTING.md
2019-11-25 10:12:59 -05:00
Drew Banin
08bb956a20
Merge pull request #1952 from fishtown-analytics/fix/bq-catalog-gen-query
...
Fix for catalog generation with external tables (bigquery)
2019-11-25 10:12:23 -05:00
Drew Banin
c4938e6567
change fallback column name from "null" to "unknown"
2019-11-25 09:35:06 -05:00
Drew Banin
0dc34c347d
Update CONTRIBUTING.md
2019-11-25 09:21:53 -05:00
Drew Banin
faf5a88727
Update CHANGELOG.md
2019-11-24 19:23:15 -05:00
Drew Banin
97722bef03
Update CHANGELOG.md
2019-11-24 16:01:12 -05:00
Drew Banin
a268f79878
( #1950 ) Coalesce BQ types and names for external tables
2019-11-22 17:00:29 -05:00
Jacob Beck
45a155a765
Use psycopg2-binary package on all platforms
2019-11-22 13:42:10 -07:00
Jacob Beck
7ae632d33f
Merge pull request #1946 from fishtown-analytics/fix/catalog-partially-quoted-resources
...
Fix catalog generation with unquoted resources on postgres/redshift (#1943 )
2019-11-22 11:59:32 -07:00
Jacob Beck
582e8e183c
just ignore it if we get multiple databases on redshift/postgres, and pick the first one instead
2019-11-21 11:13:05 -07:00
Jacob Beck
c28f2dc5c1
fix snowflake test
2019-11-20 15:09:01 -07:00
Jacob Beck
0b18212e69
Agate fixes
...
Restrict agate date+datetime formats
remove timedelta
When column types are specified for a seed, parse those columns as Text unconditionally
2019-11-20 13:53:08 -07:00
Jacob Beck
67d499e9de
Merge pull request #1936 from fishtown-analytics/fix/postgres-mixedcase-database
...
Fix postgres mixedcase database (#1800 )
2019-11-20 07:27:48 -07:00
Jacob Beck
46c28405ec
update changelog
2019-11-19 14:20:21 -07:00
Jacob Beck
2e825963f1
Merge pull request #1938 from fishtown-analytics/fix/loosen-iam-restrictions
...
Fix credential validation on iam auth (#1925 )
2019-11-19 14:16:25 -07:00
Jacob Beck
b8858cc1f5
Redshift: Make iam_duration_seconds and password optional again for "iam"
...
Move password field to be the last postgres non-optional
- now redshift can override it to be optional
Add a check for redshift "database" method that password is set
Fix up some optional/default behavior
Unit test to make sure we don't regress
2019-11-19 11:24:33 -07:00
Jacob Beck
77252de54c
add test + new db configuration stuff
2019-11-19 10:52:21 -07:00
Jacob Beck
2869467b4f
handle cased databases properly on pg/rs
2019-11-19 10:05:31 -07:00
Connor McArthur
099adf9bab
Bump version: 0.15.0rc1 → 0.15.0rc2
2019-11-19 11:08:23 -05:00
Jacob Beck
31e49fb662
Merge pull request #1935 from fishtown-analytics/feature/optional-quote-columns
...
make column quoting optional (#1917 )
2019-11-19 09:06:30 -07:00
Jacob Beck
3405745182
PR feedback
2019-11-19 06:59:36 -07:00
Drew Banin
09f2aae866
Merge pull request #1931 from fishtown-analytics/fix/drop-backup-tables-on-full-refresh
...
Fix for drop statement which was not namespaced to a schema
2019-11-18 23:27:47 -05:00
Drew Banin
0579c49ab0
Merge pull request #1932 from fishtown-analytics/fix/remove-jsonschema-requirement-from-core
...
rm jsonschema requirement from core
2019-11-18 19:06:01 -05:00
Drew Banin
7df980b5d4
Merge pull request #1934 from fishtown-analytics/fix/get-catalog-requires-one-database
...
Fix for broken catalog generation on pg/redshift
2019-11-18 17:58:07 -05:00
Jacob Beck
c2b2f70a69
make column quoting optional
2019-11-18 15:26:00 -07:00
Drew Banin
5c723bf92d
pr feedback
2019-11-18 17:16:47 -05:00
Drew Banin
905f634781
( #1926 ) Fix for broken catalog generation on pg/redshift
2019-11-18 16:40:06 -05:00
Drew Banin
b62f0ee511
Update CHANGELOG.md
2019-11-18 16:08:37 -05:00
Drew Banin
ba111db2a8
Update CHANGELOG.md
2019-11-18 16:06:20 -05:00
Drew Banin
03d6b40afe
rm jsonschema requirement from core
2019-11-18 15:28:34 -05:00
Drew Banin
6f1df6ae65
Fix for drop statement which was not namespaced to a schema
2019-11-18 14:58:00 -05:00
Drew Banin
2d6fc280fe
Merge pull request #1927 from fishtown-analytics/fix/stdout-logging-rpc
...
fix for stdout logging of on-run- hooks
2019-11-18 14:12:24 -05:00
Drew Banin
e7ffdcd09f
Merge pull request #1928 from fishtown-analytics/fix/anonymous-event-tracking-errors
...
fix for serialization issue around node timing
2019-11-18 14:12:12 -05:00
Drew Banin
17dacd31f4
fix for stdout logging of on-run- hooks
2019-11-18 13:05:02 -05:00
Drew Banin
f7b9874a93
fix for serialization issue around node timing
2019-11-18 12:45:51 -05:00
Drew Banin
36fa9bd8bf
Merge pull request #1852 from fishtown-analytics/lma-changelog
...
Update CHANGELOG for Louisa May Alcott release
2019-11-17 23:31:38 -05:00
Drew Banin
51372897bd
Update CHANGELOG.md
2019-11-17 23:26:35 -05:00
Drew Banin
1831acbaec
Update CHANGELOG.md
2019-11-17 22:49:08 -05:00
Drew Banin
4dffd524e9
Merge pull request #1923 from fishtown-analytics/bump/docs-0150
...
bump index.html for 0.15.0 in docs site
2019-11-17 20:26:16 -05:00
Drew Banin
a5a78e6173
bump index.html for 0.15.0 in docs site
2019-11-17 17:58:07 -05:00
Jacob Beck
51bca114a0
Merge pull request #1918 from JusLarsen/fix/dbname-case-sensitivity
...
remove case sensitivity from postgres/redshift dbname verification
2019-11-13 11:38:06 -07:00
Justin Larsen
72f0559f8a
remove case sensitivity from postgres dbname verification
2019-11-13 09:29:18 -07:00
Jacob Beck
9be47e67d9
Merge pull request #1903 from JusLarsen/fix/schema-test-error
...
Fix tuple index out of range in custom schema tests
2019-11-13 06:34:08 -07:00
Justin Larsen
5190b65b14
add error for empty test result set
2019-11-12 12:28:20 -07:00
Drew Banin
f7a92603c1
Merge pull request #1912 from fishtown-analytics/fix/logging-tweaks-2
...
Tweak rpc logging settings
2019-11-12 09:50:46 -05:00
Drew Banin
0c6bc3ec9b
pr review
2019-11-11 20:05:07 -05:00
Jacob Beck
53af0233a7
add tests, make log default to True, make the uid context bigger
2019-11-11 12:13:37 -07:00
Drew Banin
9c9d261433
pep8
2019-11-11 12:08:22 -07:00
Drew Banin
f64ddd6fcb
Tweak rpc logging settings
...
- respect logs_start parameter even when tasks have completed
- fix for incorrect hook index for on-run-end hooks
- clean up on-run hook log messages
- include SQL run in the context of a hook in the hook log messages
2019-11-11 12:08:22 -07:00
Jacob Beck
c26cf1977c
Merge pull request #1909 from fishtown-analytics/feature/honor-threads-flag
...
add threads to commands (#1897 )
2019-11-11 07:49:27 -07:00
Jacob Beck
abc766280a
Merge pull request #1910 from fishtown-analytics/fix/changelog-update
...
update changelog to reflect 0.14.3 and 0.14.4
2019-11-08 17:28:22 -07:00
Jacob Beck
2ca4c1a344
Merge pull request #1885 from tjengel/add_auto_dist_style
...
Add auto dist style
2019-11-08 17:23:08 -07:00
Jacob Beck
58b2955cf1
Merge pull request #1851 from kconvey/kms-encryption
...
Add a configuration option `kms_key_name` to dbt_project.yml for BigQ…
2019-11-08 14:50:03 -07:00
Jacob Beck
a75a22d71a
update changelog to reflect 0.14.4
2019-11-08 14:14:07 -07:00
tjengel
d245bcbc6e
Merge remote-tracking branch 'upstream/dev/louisa-may-alcott' into add_auto_dist_style
2019-11-08 15:02:42 -06:00
Jacob Beck
6ab537db6e
add threads to commands
...
Add tests accordingly
2019-11-08 13:32:22 -07:00
Jacob Beck
94009584ec
Merge pull request #1907 from fishtown-analytics/fix/no-deepcopy-manifest
...
Don't deep copy the manifest unless we're in single threaded mode (#1904 )
2019-11-08 12:30:50 -07:00
Jacob Beck
dc0e10f6a7
Don't deep copy the manifest unless we're in single threaded mode
...
Move environment variable handling into dbt.flags so it's easier to check
2019-11-08 12:03:19 -07:00
Jacob Beck
235ec1e3d1
Merge pull request #1901 from fishtown-analytics/fix/warehouse-to-snowflake-warehouse
...
rename "warehouse" config key to "snowflake_warehouse" (#1899 )
2019-11-08 09:40:29 -07:00
Kurt Convey
83de6b1617
Get latest changes. Merge branch 'dev/louisa-may-alcott' of https://github.com/fishtown-analytics/dbt into kms-encryption
2019-11-08 09:40:14 -07:00
Drew Banin
1f1d10033f
Merge pull request #1902 from fishtown-analytics/feature/run-sql-rpc-perf
...
fix for manifest reload logic
2019-11-08 11:10:40 -05:00
Jacob Beck
9f2a707b2b
update snowflake connection to handle new closed state properly, rename "warehouse" config key to "snowflake_warehouse", tests
2019-11-08 07:29:53 -07:00
Drew Banin
08cb38b342
fix for manifest reload logic
2019-11-07 19:11:51 -05:00
Jacob Beck
6ac347813a
Merge pull request #1900 from fishtown-analytics/feature/hook-rpc-logging
...
log hook info in json logs (#1890 )
2019-11-07 15:26:07 -07:00
Jacob Beck
b9b4ce30e8
PR feedback
2019-11-07 15:01:19 -07:00
Jacob Beck
f1f14d237e
log hook info in json logs
2019-11-07 13:36:42 -07:00
Kurt Convey
563dfc1371
Merge branch 'dev/louisa-may-alcott' of https://github.com/fishtown-analytics/dbt into kms-encryption
2019-11-07 12:47:17 -07:00
Connor McArthur
6198584f7d
Merge branch 'dev/louisa-may-alcott' of github.com:fishtown-analytics/dbt into dev/louisa-may-alcott
2019-11-07 12:59:06 -05:00
Jacob Beck
0003d57163
Merge pull request #1898 from fishtown-analytics/fix/psycopg-binary-optout
...
pin psycopg2 (#1221 )
2019-11-07 10:40:51 -07:00
Connor McArthur
5e7f718df3
Bump version: 0.15.0b2 → 0.15.0rc1
2019-11-07 10:27:33 -05:00
Jacob Beck
8bdc6877c0
pin psycopg2, add an env variable for opting out of binary
...
Set an env var so you can override the choice
2019-11-07 08:26:10 -07:00
Jacob Beck
d7eee61dd4
Merge pull request #1888 from fishtown-analytics/fix/column-quoting
...
Fix quoting on columns for seeds/incremental models (#1847 )
2019-11-05 14:14:27 -07:00
Jacob Beck
e0d4e9392a
re-case all the snowflake seed column names
2019-11-05 13:38:42 -07:00
Jacob Beck
7f8e198074
Fix quoting on columns for seeds/incremental models
...
Added tests
Also fixed quoting on column expansions
2019-11-05 09:22:06 -07:00
tjengel
e1b040c8f9
Add auto as diststyle
2019-11-04 21:56:34 -06:00
Jacob Beck
2d0f93850c
Merge pull request #1886 from fishtown-analytics/feature/test-python-3.8
...
Test python 3.8
2019-11-04 15:23:49 -07:00
Jacob Beck
610c02abc4
add some arbitrary-ish classifiers (importantly: the versions we support)
2019-11-04 14:58:47 -07:00
Jacob Beck
452cc6e78a
point this branch at the docker image it builds
2019-11-04 14:58:47 -07:00
Jacob Beck
642ebeef52
Add more versions to the dockerfile, make it build more modular-ly
...
use a PPA to get python versions instead of compiling them all
add python 3.8 tests to tox.ini and circleci.yml
update psycopg2 to 2.8.4 since they build binary wheels when possible now
2019-11-04 14:58:45 -07:00
Jacob Beck
31ca9a1041
Merge pull request #1881 from fishtown-analytics/fix/bigquery-case-sensitive
...
Fix bigquery case sensitive caching issue (#1810 )
2019-11-04 13:41:25 -07:00
Jacob Beck
670c26bdbd
use exists_ok, not_found_ok, and delete_contents on bigquery dataset operations
2019-11-04 12:23:07 -07:00
Jacob Beck
1bbc00c346
handle funky case bigquery models, add tests
2019-11-04 12:23:07 -07:00
Jacob Beck
ca4979099c
so much for being a SQLAdapter
2019-11-04 12:23:07 -07:00
Jacob Beck
30cd27e5fc
Fix BQ list_relations_without_caching + check_schema_exists
2019-11-04 12:23:07 -07:00
Jacob Beck
72d83f988e
Make bigquery adapters SQLAdapters
...
Implement more things via macros
Refactor Relations vs InformationSchemas to handle BQ better
Fix a bug where bigquery cached uppercase schema names wrong
- by using information_schema this just goes away :)
2019-11-04 12:23:07 -07:00
Jacob Beck
c4cd4fc42f
Merge pull request #1864 from fishtown-analytics/feature/query-comments
...
inject query comments (#1643 )
2019-11-04 12:22:26 -07:00
Jacob Beck
b56d93b909
if the comment macro is null/empty, no comments
2019-11-04 10:43:35 -07:00
Jacob Beck
ab9fcb4624
pin urllib3 to a version that snowflake supports
2019-11-04 09:01:34 -07:00
Jacob Beck
84d585c14c
macro support, tests, add yet another mypy env for development
2019-11-04 09:01:06 -07:00
Jacob Beck
1873f4018d
move macro stuff around
2019-11-04 09:01:06 -07:00
Jacob Beck
57f4221fc9
use block-style comments per PR feedback
...
Fix the tests
2019-11-04 09:01:06 -07:00
Jacob Beck
15ff08dc6f
less query commenting
2019-11-04 09:01:06 -07:00
Jacob Beck
5b6586de3c
Move query comments into the project config
...
Add special handling to 'dbt debug' for this behavior
Rework the dependencies/requirements for adapters since they now require more of a config object
tests...
2019-11-04 09:01:06 -07:00
Jacob Beck
ff158b8353
inject query comments
...
Make a fake "macro" that we parse specially with a single global context
Macro takes an argument (the node, may be none)
Users supply the text of the macro in their 'user_config' under a new 'query_comment'
No macros available
query generator is an attribute on the connection manager
- has a thread-local comment str
- when acquiring a connection, set the comment str
new 'connection_for' context manager: like connection_named, except also use the node to set the query string
Updated unit tests to account for query comments
Added a hacky, brittle integration test
- log to a custom stream and read that
Trim down the "target" context value to use the opt-in connection_info
- Make sure it contains a superset of the documented stuff
- Make sure it does not contain any blacklisted items
Change some asserts to raise InternalExceptions because assert error messages in threads are useless
2019-11-04 09:01:06 -07:00
Jacob Beck
f985902a00
Merge pull request #1878 from fishtown-analytics/feature/run-operation-snapshot-rpc
...
add run-operation + snapshot to the RPC server (#1875 )
2019-11-01 13:44:46 -06:00
Drew Banin
e022e73b0c
Merge pull request #1876 from clausherther/add/quote-accepted-values
...
Adds "quote" parameter to "accepted_values" test
2019-11-01 14:43:39 -04:00
Jacob Beck
658be46831
pin dependencies of dependencies
2019-11-01 10:14:25 -06:00
Jacob Beck
06018ee3b5
PR feedback
2019-11-01 09:22:57 -06:00
Jacob Beck
1b4c76365c
add run_operation + snapshot to the RPC server
2019-10-30 14:30:37 -06:00
Jacob Beck
8239f9c4e3
Merge pull request #1872 from fishtown-analytics/fix/bigquery-snowflake-unknown-relation-types
...
Fix bigquery snowflake unknown relation types
2019-10-30 11:03:40 -06:00
Claus Herther
239e6738a8
Adjusts yaml idents
2019-10-29 10:20:23 -07:00
Claus Herther
d605ff6fc2
Adds quote parameter to accepted_values schema test
2019-10-29 10:13:12 -07:00
Jacob Beck
76dc41b86e
Merge pull request #1871 from fishtown-analytics/fix/log-rotation
...
rotate the file at 10mb instead of by date
2019-10-29 07:30:46 -06:00
Jacob Beck
2f85d83090
Allow for unknown relation types in more places
...
For bigquery:
if a relation type is not known, use 'RelationType.External' in place of None
For snowflake/other sql adapters:
if the relation type returned by list_relations_without_caching macro is not
a known RelationType, treat it as External
2019-10-28 15:40:59 -06:00
Jacob Beck
1f34139df3
on bigquery, default to External instead of None when we do not recognize the table type
2019-10-28 15:14:53 -06:00
Jacob Beck
78115cf12a
rotate the file at 10mb instead of by date
2019-10-28 14:59:01 -06:00
Jacob Beck
3a030ab74b
Merge pull request #1867 from fishtown-analytics/fix/update-hologram-005
...
Updated hologram to handle parallel json_schema
2019-10-28 14:58:01 -06:00
Jacob Beck
ef3d63f30c
Updated hologram to handle parallel json_schema
2019-10-28 08:13:30 -06:00
Kurt Convey
0d8e0cb1ca
Merge branch 'kms-encryption' of https://github.com/kconvey/dbt into kms-encryption
2019-10-24 10:13:06 -06:00
Kurt Convey
a8ee33ff08
Sync latest version
...
Merge branch 'dev/louisa-may-alcott' of https://github.com/fishtown-analytics/dbt into kms-encryption
2019-10-24 10:11:36 -06:00
Kurt Convey
16f93ceee5
Sync latest version
...
Merge branch 'dev/louisa-may-alcott' of https://github.com/fishtown-analytics/dbt into kms-encryption
2019-10-24 10:09:33 -06:00
Jacob Beck
b3ef028361
Merge pull request #1859 from fishtown-analytics/feature/top-level-poll-timings
...
Feature: poll timing information, status -> state
2019-10-23 17:44:57 -06:00
Jacob Beck
672363bff4
Provide consistent timings between poll and ps results
...
Also rename status -> state in results
Add tests
2019-10-23 17:08:39 -06:00
Jacob Beck
a1d651aed2
Merge pull request #1855 from fishtown-analytics/fix/corrupted-task-state
...
Fix corrupted task state
2019-10-23 15:47:25 -06:00
Jacob Beck
5fa24d7880
Merge pull request #1856 from fishtown-analytics/fix/no-normcase-filenames
...
remove os.path.normcase calls
2019-10-23 15:37:59 -06:00
Jacob Beck
78d23c8243
Merge pull request #1858 from fishtown-analytics/fix/re-add-metadata
...
fix the `metadata` behavior for dbt-utils (#1857 )
2019-10-23 15:37:50 -06:00
Jacob Beck
d43d49494d
Merge pull request #1837 from fishtown-analytics/feature/dbt-deps-rpc
...
Add a "dbt deps" api call (#1834 )
2019-10-23 15:21:46 -06:00
Jacob Beck
f4ca94f6d8
Add a special handler to support the dbt_utils._is_relation macro
2019-10-23 14:07:08 -06:00
Jacob Beck
0dedac6a2e
remove normcase calls
...
add tests for shouting model names
2019-10-23 13:46:29 -06:00
Jacob Beck
f268ee9ae9
Make builtins just another kind of RemoteMethod
...
Move GC into its own object
Make parsing/parse failed into special callables and return those
2019-10-23 12:54:38 -06:00
Jacob Beck
57aa443bd7
fix an rpc-only deps bug, fix some stack traces going to stderr
2019-10-23 12:53:38 -06:00
Jacob Beck
45bc25d589
PR feedback
...
Fix a bug where failed deps were "lost" forever
Un-support 'cli_args' + 'deps', skip those tests
Add a new test suite for RPC tests
Move flaky RPC tests into new suite where they passes like they should
2019-10-23 09:07:00 -06:00
Drew Banin
c98da40873
Update CHANGELOG.md
2019-10-23 10:24:37 +01:00
Kurt Convey
bcd953490d
Add a configuration option kms_key_name to dbt_project.yml for BigQuery
...
projects where a Cloud KMS key can be specified and used to encrypt
models. The key is specified to bigquery in the ddl OPTIONS.
2019-10-22 14:16:41 -06:00
Connor McArthur
3103442447
Merge pull request #1849 from fishtown-analytics/use-setuptools
...
Use setuptools instead of distutils.core
2019-10-22 12:37:30 -04:00
Connor McArthur
e08bc540cf
use setuptools instead of distutil.core
2019-10-22 12:28:26 -04:00
Connor McArthur
d6682f092c
Bump version: 0.15.0b1 → 0.15.0b2
2019-10-22 12:18:08 -04:00
Jacob Beck
bf5a6b84b4
fix the dbt deps cleanup
...
dbt clean does not clean the modules path
shutil.rmtree does :)
bump test timeouts
SIGHUP also reloads the config
2019-10-21 14:15:51 -06:00
Jacob Beck
97e0cdebbc
Merge pull request #1845 from fishtown-analytics/fix/more-packaging-issues
...
More packaging fixes
2019-10-21 13:20:51 -06:00
Jacob Beck
6f64f51f8d
normalize comparisons
2019-10-21 11:05:58 -06:00
Jacob Beck
f94a7a73c1
More packaging fixes
...
- Add both 'dbt' and 'dbt.*' to the namespace package search path
- set zip_safe=False, because I think zipping breaks macros
- Make CI tests use non-editable installs
- make tox use editable mode for explicit-* so development is still possible without rebuilding tox
- remove {toxinidir} stuff that breaks core installation
- as of tox 1.6.1, requirements executes within toxinidir.
- fix paths to account for installing things
2019-10-21 09:52:29 -06:00
Jacob Beck
5cd8cff11d
dbt deps now blocks incoming API calls, reloads the config, runs deps, and builds a new manifest
2019-10-20 08:40:33 -06:00
Jacob Beck
c293fd94d3
fix a couple tests
2019-10-18 17:38:52 -06:00
Jacob Beck
91f0e444c0
Add Kill/Fail task states + handling for them
...
Move setting real_task into set_args on the CLI handler
Move set_args into the handle() method, and out of task bootstrap
- now it happens in the parent/handling process
- now the parent process can know what real task was chosen
Make the cli handler report status properly
2019-10-18 17:38:52 -06:00
Jacob Beck
abdcdefb5f
Add a "dbt deps" api call
...
Refactor deps
- split it up
- move most of it into dbt.deps
Add the idea of a remote method that does not require a manifest
- it does still require config/args
- set up the various reloading logic to not care about reloading these methods
Add dbt deps api call
- also available as if it were from the cli
Add tests
2019-10-18 17:38:32 -06:00
Jacob Beck
6164733489
Merge pull request #1841 from fishtown-analytics/fix/namespace-package-builds
...
Fix namespace package builds
2019-10-18 15:59:20 -06:00
Jacob Beck
f9a0ccf318
Fix namespace package builds
...
use find_namespace_packages() instead of find_packages()
2019-10-18 15:18:02 -06:00
Connor McArthur
bb82093ffc
Bump version: 0.15.0a1 → 0.15.0b1
2019-10-18 14:00:59 -04:00
Drew Banin
d3139a531c
Merge pull request #1840 from fishtown-analytics/fix/logging-tweaks
...
change info log lines to debug logs
2019-10-18 13:53:12 -04:00
Drew Banin
3a40cf2373
change info log lines to debug logs
2019-10-18 13:20:07 -04:00
Jacob Beck
a7b0d05d3c
Merge pull request #1839 from fishtown-analytics/fix/snowflake-query-bad-errors
...
Handle a number of bad failure-path behaviors (#1807 )
2019-10-17 14:42:37 -04:00
Jacob Beck
ae796a8497
Handle a number of bad failure-path behaviors
...
- When a connection is missing, raise a special exception
- instead of RuntimeError, so we catch it better
- Be graceful if a connection does not exist during nice_connection_name
- Handle the case where exceptions caught by the Snowflake exception handler do not have a 'msg' attr
- Re-raise exceptions in the adapter exception handlers "from" the originating error
2019-10-17 07:42:00 -06:00
Jacob Beck
6d44caa4e3
Merge pull request #1836 from fishtown-analytics/feature/partial-parse-upgrades
...
Partial parse updates (#1835 )
2019-10-16 09:08:45 -04:00
Jacob Beck
f32519b156
Partial parse upates
...
Use the user's profile config and the CLI arguments to determine whether to try partial parsing
- CLI wins
Check the dbt version number as part of deciding if a cached result is ok
2019-10-15 18:25:13 -06:00
Jacob Beck
d5824d9238
Merge pull request #1833 from fishtown-analytics/fix/postgres-external-materialized-views
...
postgres: gracefully handle materialized views (#1698 )
2019-10-15 19:14:32 -04:00
Drew Banin
e83aab202b
Merge pull request #1682 from fishtown-analytics/fix/minimize-incremental-downtime
...
(#525 ) drop existing relation at end of full-refresh incremental build
2019-10-15 14:44:43 -04:00
Drew Banin
00a22e18cc
Merge pull request #1795 from fishtown-analytics/fix/bq-catalog-query
...
(#1576 ) use the information schema on BigQuery
2019-10-15 14:44:18 -04:00
Jacob Beck
fee9382c7f
Properly ignore materialized views
...
add_link now creates a relation if it doesn't exist (as "external")
add/fix tests
2019-10-15 11:58:23 -06:00
Drew Banin
f757a08a99
improve the BigQuery Relations understanding of the information schema
2019-10-15 12:28:09 -04:00
Drew Banin
414716b841
pr feedback
2019-10-15 11:32:27 -04:00
Drew Banin
50f4f8a5b0
pr feedback
2019-10-15 11:31:09 -04:00
Jacob Beck
73d0308e35
Merge pull request #1831 from fishtown-analytics/feature/avoid-osx-flaky-rpc
...
Fix for issues with some Mojave installs
2019-10-15 09:43:18 -04:00
Drew Banin
43b8293a07
pep8; code cleanup
2019-10-14 23:13:09 -04:00
Drew Banin
15c26192e8
Include Location field in catalog from the SCHEMATA table
2019-10-14 23:07:42 -04:00
Drew Banin
95a0587499
handle changing partition/cluster configs on BQ
2019-10-14 22:48:52 -04:00
Drew Banin
0f1693a9d7
( #525 ) drop existing relation at end of full-refresh incremental build
2019-10-14 21:54:22 -04:00
Drew Banin
4c624d0a85
( #1576 ) use the information schema on BigQuery
2019-10-14 21:42:25 -04:00
Jacob Beck
6287d6d30b
Merge pull request #1830 from fishtown-analytics/feature/improve-rpc-compile-performance
...
Feature/improve rpc compile performance (#1824 )
2019-10-14 18:59:53 -04:00
Jacob Beck
c19260ec4e
Speculative fix for issues with some Mojave installs (python 3.7-related?)
...
Use an internal multiprocessing.context in dbt.flags for multiprocessing things
- On osx, force the use of "spawn" to handle https://bugs.python.org/issue33725
- Bring _nt_setup back from the dead as _spawn_setup
2019-10-14 16:42:09 -06:00
Jacob Beck
43daea05c1
remove inscrutable __getattr__ override
2019-10-14 16:28:28 -06:00
Jacob Beck
75916754a6
add register_adapter to dbt debug
2019-10-14 13:12:28 -06:00
Jacob Beck
14d8683135
Avoid deepcopying the manifest 1x per task per sighup
2019-10-14 13:02:13 -06:00
Jacob Beck
61f8e6d4a1
fix unit tests
2019-10-14 12:05:12 -06:00
Jacob Beck
b658f879f9
fix field serialization for hologram+mypy
2019-10-14 12:05:12 -06:00
Jacob Beck
4aa4295508
Make mypy totally happy
...
Some circular import cleanups
remove is_type function, just compare to resource_type
Add type checking for dbt deps
2019-10-14 12:05:12 -06:00
Jacob Beck
75c8feaeb9
Make ManifestMetadata a first-class object
2019-10-14 12:05:12 -06:00
Jacob Beck
66ff79dfbd
Refactor RPC tests
2019-10-14 12:05:12 -06:00
Jacob Beck
7206c202bf
Manifests are only parsed, not compiled at SIGHUP/startup time
...
The _sql tasks now compile any ref'ed CTE chains at RPC call time
Give RPC tasks their own folder
- task/rpc_server -> task/rpc/server
- task/remote -> task/rpc/{project_commands,sql_commands,base}
Linker enhancements:
- Expose subset graph building so multiple methods can use it
- Expose a way for the linker to provide an interable of the ephemeral ancestors of a node
- it's guaranteed to be ordered (so nested CTEs behave)
2019-10-14 11:37:40 -06:00
Jacob Beck
773c979955
Refactor internal RPC logic to support just getting parsed manifests
...
RemoteCallableResult -> RPCResult
RemoteCallable -> RemoteMethod
- move some things from RPCTask -> RemoteMethod
- recursive_subclasses classmethod
things in core/dbt/rpc now are all based on RemoteMethods, not RPCTasks
2019-10-14 11:37:37 -06:00
Jacob Beck
ef16a99f88
Refactors for mypy:
...
initial refactoring of adapter factory stuff
Move HasCredentials protocol into connection contract and use that in the base connection
2019-10-14 11:37:33 -06:00
Jacob Beck
d86092ae78
loader.GraphLoader -> parser.manifest.ManifestLoader
...
Create new helper function dbt.perf_utils.get_full_manifest
Update task.runnable accordingly
Update RPC server accordingly
2019-10-14 11:37:30 -06:00
Jacob Beck
6c87bed66b
Add manifest.expect method
2019-10-14 11:37:26 -06:00
Drew Banin
2d100c33b8
Merge pull request #1823 from heisencoder/fix/printer-log-level
...
update logging levels in printer.py to match the intent
2019-10-11 15:39:59 -04:00
Jacob Beck
81b755ff0f
Merge pull request #1828 from fishtown-analytics/feature/task-tags
...
Add task tags parameter (#1822 )
2019-10-11 14:42:09 -04:00
Jacob Beck
fb9747b6be
Add task tags parameter
...
Create base class for parameter types
- include arbitrary "task_tags" parameter
- include "timeout" so we can extract/document full/valid json schemas
- this includes some extra-fiddly behavior around numbers
Add output to ps and poll results (including errors)
Fix a number of type annotation issues along the way
2019-10-11 11:51:44 -06:00
Drew Banin
cd1eccf5f4
Merge pull request #1826 from fishtown-analytics/fix/bq-result-type-inference
...
fix for incorrect type inference on bigquery
2019-10-10 22:39:49 -04:00
Drew Banin
3b69d310dd
( #1825 ) fix for incorrect type inference on bigquery
2019-10-10 16:26:15 -04:00
Matt Ball
dc2ebf292b
update logging levels in printer.py to match the intent
...
See https://github.com/fishtown-analytics/dbt/issues/1818
2019-10-10 10:28:03 -06:00
Jacob Beck
2815b33b0a
Merge pull request #1821 from fishtown-analytics/fix/json-compatible-regexes
...
jsonschema+deps fixes
2019-10-10 12:08:37 -04:00
Jacob Beck
9819c3a01e
Merge pull request #1806 from fishtown-analytics/feature/transactional-json-logging
...
Transactional json logging
2019-10-10 09:22:40 -04:00
Jacob Beck
cf619eacc5
Merge pull request #1812 from fishtown-analytics/feature/cli-args-rpc
...
Add cli execution support (#1811 )
2019-10-10 09:22:01 -04:00
Jacob Beck
410086df6b
jsonschema+deps fixes
...
Fix the offending regex to use js-compatible syntax
Fix the werkzeug pin
Pin hologram to 0.0.4
Pin dataclasses out of an excess of caution
2019-10-10 06:58:42 -06:00
Jacob Beck
2b9c5ae7d4
PR feedback
...
s/model/node/
include resource_type
2019-10-10 05:22:36 -06:00
Jacob Beck
eb921e9281
Merge pull request #1813 from fishtown-analytics/fix/custom-snapshot-strategies
...
Fix custom snapshot strategy support (#1794 )
2019-10-10 07:11:06 -04:00
Jacob Beck
6d2aca182c
Merge pull request #1816 from fishtown-analytics/feature/test-nodes-include-name-kwargs
...
add test metadata for schema tests, add test tests (#1154 )
2019-10-10 07:10:43 -04:00
Jacob Beck
a2691d5dad
add test metadata for schema tests, add test tests
2019-10-08 14:01:38 -06:00
Jacob Beck
c1b7c4c1b7
Merge pull request #1814 from fishtown-analytics/feature/upgrade-deps
...
Upgrade dependencies
2019-10-08 13:19:04 -04:00
Jacob Beck
66ffcf1e1b
Upgrade dependencies
...
Networkx minimum version set to 2.3, dropped 1.x support
- removed a backwards compatibility fix
Give colorama a range of supported versions, preserving the old minimum
Give sqlparse a range of supported versions, preserving the old minimum
Bump werkzeug to 0.15.x, 0.16 also works so allow that
- when 1.0 releases it looks like it will work as well, but wait on that
- drop 0.14.x support
Set an upper bound on snowflake-connector-python of 2.0.x
2019-10-08 10:43:35 -06:00
Jacob Beck
e2d9eb4772
Fix custom snapshot strategy support, add tests
2019-10-08 09:52:20 -06:00
Jacob Beck
0bbaa5be21
deal with MRO issues by making another intermediate class
2019-10-08 08:01:37 -06:00
Drew Banin
54d0230d1b
Merge pull request #1809 from captainEli/fix/source-arguments
...
Improve Error Messaging for Invalid Source Arguments
2019-10-07 21:24:33 -04:00
Jacob Beck
4018e550fe
PR feedback
...
Add node count/total node to log lines
Make model status structured
2019-10-07 20:14:23 -04:00
Jacob Beck
21f2e01921
Add cli execution support
2019-10-07 19:50:12 -04:00
Eli Kastelein
34cb9c7186
Reduce line length for pep8
2019-10-07 16:06:09 -07:00
Jacob Beck
52ac98ea1c
Address feedback, appease flake8/mypy/unit tests
2019-10-07 11:58:16 -04:00
Jacob Beck
aa254b350b
Transactional logging
2019-10-07 11:58:14 -04:00
Jacob Beck
8194fd4d13
Merge pull request #1801 from fishtown-analytics/feature/add-docs-generate-rpc
...
docs generate RPC task (#1781 )
2019-10-07 11:56:51 -04:00
Jacob Beck
2031e232ca
PR feedback, also remove an extra manifest-building from "dbt docs generate"
2019-10-07 10:15:53 -04:00
Eli Kastelein
b71e51e30f
Fix typo in error message
2019-10-04 20:59:37 -07:00
Eli Kastelein
a1d93f92cd
refactor to make more similar to RefResolver
2019-10-04 20:38:59 -07:00
Eli Kastelein
3a5c0ebc78
SourceResolver call to use *args
2019-10-04 20:31:03 -07:00
Jacob Beck
5061397e66
docs generate RPC task
2019-10-02 17:38:43 -04:00
Jacob Beck
31e085b7df
Merge pull request #1780 from fishtown-analytics/refactor/relations-as-jsonschemamixins
...
Remove APIObject (#1762 )
2019-10-02 11:10:41 -06:00
Jacob Beck
13a7d96d84
Merge pull request #1798 from fishtown-analytics/feature/rename-rpc-methods
...
rename RPC methods (#1779 )
2019-10-02 09:01:07 -06:00
Jacob Beck
3765435ee6
Merge pull request #1791 from fishtown-analytics/feature/log-format-parameter
...
add log-format flag (#1237 )
2019-10-02 09:00:33 -06:00
Jacob Beck
e61f08b981
rename RPC methods
2019-10-02 08:28:04 -06:00
Jacob Beck
1fbd82d19c
Merge pull request #1793 from fishtown-analytics/feature/merge-0.14.3-louisa-may-alcott
...
Feature/merge 0.14.3 louisa may alcott
2019-10-02 07:57:26 -06:00
Jacob Beck
dfb4b3a2c8
PR feedback
...
add typing extensions module to setup.py
Update changelog
2019-10-02 07:13:16 -06:00
Drew Banin
5f30d5ddaf
Merge pull request #1787 from darrenhaken/dev/louisa-may-alcott
...
Add trailing comma when creating BQ options to each list item
2019-10-01 10:39:52 -04:00
Jeremy Cohen
aceda5ec62
Merge pull request #1784 from fishtown-analytics/feature/external-source-config
...
Feature/external source config
2019-09-27 16:51:47 -04:00
Jacob Beck
626e642d90
PR feedback
2019-09-27 14:50:47 -06:00
Jacob Beck
812c549156
Merge pull request #1788 from fishtown-analytics/feature/warehouse-model-config
...
Add runtime per-model warehouse config on snowflake models (#1358 )
2019-09-27 14:13:05 -06:00
Jacob Beck
34ca230473
Merge branch 'dev/0.14.3' into dev/louisa-may-alcott
2019-09-27 13:53:36 -06:00
Jacob Beck
c211d0e436
Merge pull request #1792 from fishtown-analytics/fix/run-operation-flat-graph
...
Make run-operation a ManifestTask
2019-09-27 13:52:06 -06:00
Jacob Beck
52f62430f4
remove extra transaction logic, not needed
2019-09-27 13:45:04 -06:00
Jacob Beck
6fdac5020f
Call build_flat_graph everywhere that wants it
...
Make run-operation a ManifestTask and build a flat graph so it has the "graph" context member
Make sure the RPC server task calls build_flat_graph in __init__
2019-09-27 13:26:05 -06:00
Jacob Beck
dca02111be
add log-format flag
2019-09-27 11:22:26 -06:00
Jeremy Cohen
fcd86e3298
Address failing docs int tests
2019-09-27 13:22:07 -04:00
Jacob Beck
286753b464
Merge pull request #1783 from fishtown-analytics/feature/seeds-path-resolution
...
Fix the seed path definition
2019-09-27 11:06:35 -06:00
Jacob Beck
a227b31162
Add runtime per-model warehouse config on snowflake models
...
Add warehouse config to snowflake
Add concept of adapter-level pre/post model hooks
Use those hooks to optionally set a warehouse on a per-model basis
Added some integration tests
- one tests that the warehouse is applied by setting a bad one and failing
- one that tests that overriding the warehouse to a good one is ok
2019-09-27 10:45:23 -06:00
Jacob Beck
d246bab0eb
Fix seeds in dependencies
...
Modified a test to cover the bug
Update the FilePath definition
Add a new attribute to seeds (seed_file_path)
Test fixes
Added --no-version-check flag to dbt seed
2019-09-27 10:30:45 -06:00
Jacob Beck
78a199f946
Merge pull request #1785 from fishtown-analytics/feature/hologram-use-restrict-fields
...
Use the restrict hologram metadata field instead of multiple enum types
2019-09-27 09:50:32 -06:00
Jeremy Cohen
54009a05cf
Merge branch 'dev/louisa-may-alcott' of github.com:fishtown-analytics/dbt into feature/external-source-config
2019-09-27 11:13:09 -04:00
Jacob Beck
eb9bfcda4a
Convert Relation types to hologram.JsonSchemaMixin
...
Fix a lot of mypy things, add a number of adapter-ish modules to it
Split relations and columns into separate files
split context.common into base + common
- base is all that's required for the config renderer
Move Credentials into connection contracts since that's what they really are
Removed model_name/table_name -> consolidated to identifier
- I hope I did not break seeds, which claimed to care about render(False)
Unify shared 'external' relation type with bigquery's own
hack workarounds for some import cycles with plugin registration and config p
arsing
Assorted backwards compatibility fixes around types, deep_merge vs shallow merge
Remove APIObject
2019-09-27 09:01:18 -06:00
Jacob Beck
91c51c88cd
Use the restrict hologram metadata field instead of multiple types
...
mypy stuff
Use restrict
node types
test fixes
contract tweaks
2019-09-27 09:00:42 -06:00
Jacob Beck
2a69a371fd
fix tests broken by the merge
2019-09-27 08:54:28 -06:00
Jacob Beck
2e6398d5b5
fix bad merge
2019-09-27 08:44:37 -06:00
Jacob Beck
8b622c964f
Merge branch 'dev/0.14.3' into dev/louisa-may-alcott
2019-09-27 08:37:14 -06:00
Darren Haken
fd8049c997
Add trailing comma when creating BQ options to each list item
...
- This is a bug in the existing code, BQ needs commas between option items.
Discovered it when creating a temp table using dbt_utils
2019-09-27 15:36:21 +01:00
Jeremy Cohen
2ab21ed710
Address failing tests
2019-09-26 17:46:35 -04:00
Jeremy Cohen
01be413927
Optional data_type. More rigorous subdataclass
2019-09-26 17:45:59 -04:00
Jeremy Cohen
b93563bfeb
Create AdditionalPropertiesAllowed subdataclass
2019-09-26 17:45:47 -04:00
Jeremy Cohen
c68de1066d
Support external table config
2019-09-26 17:45:38 -04:00
Jacob Beck
23484b18b7
Merge pull request #1776 from fishtown-analytics/feature/partition-filters-sources
...
Add filter field to source table definitions (#1495 )
2019-09-26 07:56:18 -06:00
Drew Banin
aea03ebba4
Update CHANGELOG.md
2019-09-25 20:49:54 -04:00
Jacob Beck
0522535a15
move filter into the freshness settings
2019-09-24 13:49:56 -06:00
Jacob Beck
2799a8c34d
Mypy fixes
...
Added some type annotations
Clean up some mypy issues around the "available" decorators
2019-09-24 13:41:27 -06:00
Jacob Beck
d22f3653b7
add filter field
2019-09-24 13:41:27 -06:00
Jacob Beck
509e0e8363
Merge pull request #1777 from fishtown-analytics/fix/cleanup-dbt-help
...
Address all the whitespace issues (#1767 )
2019-09-24 09:43:52 -06:00
Jacob Beck
1cce96f4ee
Merge pull request #1775 from fishtown-analytics/feature/validate-targets-programmatically
...
Add staticmethod on the debug task to validate targets
2019-09-24 08:14:28 -06:00
Jacob Beck
7981d8ed8c
Address all the whitespace issues
2019-09-23 14:59:46 -06:00
Jacob Beck
b78fd60343
Add validation feature, add unit + integration tests for various aspects of dbt debug
2019-09-23 09:56:06 -06:00
Jacob Beck
f9c8442260
Merge pull request #1770 from fishtown-analytics/feature/expose-cache-to-materializations
...
Expose the cache in dbt's rendering contexts (#1683 )
2019-09-23 09:54:28 -06:00
Jacob Beck
f84cbc3091
Update core/dbt/deprecations.py
...
Include the actual docs link!
Co-Authored-By: Drew Banin <drew@fishtownanalytics.com >
2019-09-23 09:18:26 -06:00
Jacob Beck
f6406c95aa
Merge pull request #1774 from fishtown-analytics/fix/include-database-comparing-catalog
...
Include the database when deciding if two tables are the same (#1708 )
2019-09-20 14:16:42 -06:00
Jacob Beck
e83f47b82e
integration tests for the deprecation
2019-09-20 14:16:00 -06:00
Jacob Beck
145e0b228e
Add support for a materialization return value
...
Materializations now return a list of relations they added
- those get added to the cache
Updated existing materializations
Added backwards compatibility support + a deprecation warning
2019-09-20 14:06:14 -06:00
Jacob Beck
ca1c84c9d4
expose the cache to macros
...
make new cache manipulation methods
mark the methods available to the sql context
move cache manipulation into appropriate macros and methods
update the changelog
fix some type checking
2019-09-20 14:05:39 -06:00
Jacob Beck
a9bb1aabbb
Include the database when deciding if two tables are the same in catalog generation
...
Convert catalog intermediate structure into something more useful
Make comparing manifests to catalogs faster by generating an explicit identifier to id mapping
Make the identifier to unique ID mapping include databases
Convert catalog to use dataclasses/hologram types
Fix unit tests to test what we actually care about
No changes to integration tests means no need to change dbt docs, hooray
2019-09-20 10:42:23 -06:00
Drew Banin
e31a9af5be
Merge pull request #1747 from Carolus-Holman/dev/louisa-may-alcott
...
Added Copy Grants to Snowflake Adapter
2019-09-18 09:51:42 -04:00
Drew Banin
c941434510
Merge pull request #1769 from fishtown-analytics/fix/snapshot-cli-output
...
Standardize snapshot CLI output
2019-09-18 09:40:59 -04:00
Jacob Beck
7a305ca456
Merge pull request #1759 from fishtown-analytics/feature/snapshot-config-block
...
snapshot config blocks (#1613 )
2019-09-18 07:24:50 -06:00
Drew Banin
e2e509b8a4
( #1768 ) standardize snapshot cli output
2019-09-17 22:37:14 -04:00
Jacob Beck
92d08a5682
run hooks in snapshots, add tests
2019-09-17 12:59:41 -06:00
Jacob Beck
0fa4523450
Merge pull request #1761 from fishtown-analytics/fix/fix-all-test-names
...
Rename integration test methods to conform to standard
2019-09-17 09:41:23 -06:00
Jacob Beck
dbcb9c562f
make all tests have the adapter in the name
...
enforce that restriction instead of warning
also shut up a name-related warning from unittest
2019-09-17 09:13:03 -06:00
Jacob Beck
aff1d1d9b3
snapshot config blocks
...
Snapshots can now be configured via {{ config(...) }}
Added tests that exercise that
2019-09-17 08:41:31 -06:00
carolush@slalom.com
226247d858
( #1744 ) add copy grants option for Snowflake
2019-09-17 08:24:59 -05:00
Jacob Beck
f9bc7c56e5
Merge pull request #1753 from fishtown-analytics/fix/merge-0.14.3
...
Fix/merge 0.14.3
2019-09-16 11:03:20 -06:00
Jacob Beck
4910d6cc9d
Merge pull request #1752 from fishtown-analytics/fix/log-cli-errors
...
Fix: log cli errors
2019-09-16 10:59:15 -06:00
Jacob Beck
036fc07500
with main() logging fixed, this now causes double-logging, so remove it
2019-09-16 10:29:31 -06:00
Jacob Beck
189c1c0e0c
Merge branch 'dev/0.14.3' into dev/louisa-may-alcott
2019-09-16 10:18:29 -06:00
Jacob Beck
017f4175f8
Merge pull request #1750 from fishtown-analytics/fix/pregenerate-flat-graph
...
Fix flat graph generation bug (#1740 )
2019-09-16 10:14:50 -06:00
Jacob Beck
8cd93f51c2
handle an annoying bug where dbt silently failed sometimes
2019-09-16 10:06:37 -06:00
Jacob Beck
cbd824ceda
Merge pull request #1751 from fishtown-analytics/fix/fail-on-parse-errors
...
Fail on parse errors (#1493 )
2019-09-16 09:49:35 -06:00
Jacob Beck
1d91890c9f
no more warnings for parse failures
2019-09-13 14:55:37 -06:00
Jacob Beck
726004bb3f
Make flat graph an attribute and build it during runtime initialization
2019-09-13 13:47:45 -06:00
Jacob Beck
cd5d39dbce
Merge pull request #1749 from fishtown-analytics/fix/close-connections-properly
...
force-cleanup all adapter connections before exiting handle_and_check (#1271 )
2019-09-13 13:38:14 -06:00
Jacob Beck
29a14cf0c3
when parsing hits an exception, ensure we actually log it
2019-09-13 12:12:26 -06:00
Jacob Beck
b83b82849f
force-cleanup all adapter connections before exiting handle_and_check
2019-09-13 11:18:50 -06:00
Connor McArthur
c4892d9f33
Merge pull request #1745 from fishtown-analytics/dev/0.14.2
...
dbt 0.14.2
2019-09-13 12:55:12 -04:00
Jacob Beck
3ebe95d669
Merge pull request #1748 from fishtown-analytics/feature/context-blacklist
...
add disallowed project names list to dbt projects (#1696 )
2019-09-13 10:44:39 -06:00
Jacob Beck
cfb55efcf7
add disallowed project names list to dbt projects
...
Raise on disallowed project names
Set defaults in project contract for testability
Add tests
2019-09-13 08:35:37 -06:00
Jacob Beck
5283002951
Merge pull request #1735 from fishtown-analytics/feature/async-rpc
...
Feature: async rpc (#1706 )
2019-09-13 07:58:48 -06:00
Connor McArthur
6e5456d14c
update date in CHANGELOG
2019-09-13 09:49:23 -04:00
Connor McArthur
5e706816e4
undo accidental werkzeug bumpversion, fix RELEASE instructions
2019-09-13 09:48:51 -04:00
Jacob Beck
2cf2c82899
add early error on invalid parameters via inspect shenanigans
2019-09-12 14:17:11 -06:00
Jacob Beck
9ea6e8d0d0
flake8
2019-09-12 14:17:11 -06:00
Jacob Beck
0563061b25
More flexible json.dumps in rpc
...
If the rpc server gets a non-serializable response component, call str() on it
2019-09-12 14:17:11 -06:00
Jacob Beck
5937c20ed1
bigquery error handling: log strings, not Exception objects
2019-09-12 14:17:11 -06:00
Jacob Beck
58fcbcf7a5
Improve logging/error handling
...
Pass werkzeug logging through
suppress all non-dbt logs in the RPC response
filter out all non-dbt/werkzeug logs below WARNING (dbt and dbt rpc)
lower all remaining non-dbt/werkzeug logs to DEBUG (dbt and dbt rpc)
- attach old_level in extras if we do
Handle some more protocol exceptions gracefully
2019-09-12 14:17:11 -06:00
Jacob Beck
34980655b2
Handle started/ended correctly
2019-09-12 14:17:11 -06:00
Jacob Beck
d51ebac71b
remove windows support from RPC
...
Also disable RPC tests on windows
managing windows idiosyncrasies is just too much
2019-09-12 14:17:11 -06:00
Jacob Beck
5e52c64153
gc, tests
2019-09-12 14:17:11 -06:00
Jacob Beck
4cb4318426
Implement async RPC (no gc of tasks)
2019-09-12 14:17:11 -06:00
Jacob Beck
d23b1aa755
Clean up return types and return values for async rpc work + mypy
...
Add __len__, __iter__, and __getitem__ to ExecutionResult/FreshnessResult
- tests and some internal callers are now happy without modifying them
2019-09-12 14:17:11 -06:00
Connor McArthur
7151ac1f81
Bump version: 0.14.1 → 0.14.2
2019-09-12 15:58:43 -04:00
Drew Banin
ee921cc0ca
Update CHANGELOG.md
2019-09-12 15:01:05 -04:00
Drew Banin
5dc776f857
Merge pull request #1743 from Carolus-Holman/feature/snowflake-secure-view-4
...
Feature/snowflake secure view 4
2019-09-12 10:29:52 -04:00
carolush@slalom.com
c4288b1e82
Added Support for Snowflake Secure Views.
2019-09-11 14:21:57 -05:00
carolush@slalom.com
5b29c19afe
Revert "Adding support for Snowflake Secure Views - {{ config(materialized='view',secure=true) }} in Model"
...
This reverts commit ac1b906888 .
2019-09-11 09:05:30 -05:00
carolush@slalom.com
ac1b906888
Adding support for Snowflake Secure Views - {{ config(materialized='view',secure=true) }} in Model
2019-09-11 08:48:52 -05:00
Jacob Beck
3e09319f8c
Merge pull request #1734 from fishtown-analytics/fix/0.14.2-disable-cache-logging
...
Fix/0.14.2 disable cache logging (#1725 )
2019-09-10 12:31:50 -06:00
Jacob Beck
1fab0aba7e
Merge pull request #1732 from fishtown-analytics/fix/dbt-cache-logging-performance
...
disable dbt cache logging by default (#1725 )
2019-09-10 11:50:02 -06:00
Jacob Beck
2d52eda730
disable dbt cache logging
2019-09-10 11:15:43 -06:00
Jacob Beck
6f51e0de13
fix flag setting
2019-09-10 11:08:22 -06:00
Drew Banin
491ad692d7
Merge pull request #1729 from fishtown-analytics/fix/0141-regression-snapshot-freshness
...
Fix/0141 regression snapshot freshness
2019-09-10 13:03:49 -04:00
Drew Banin
26f573fdc8
pr feedback
2019-09-10 11:34:52 -04:00
Drew Banin
689a0ea4cc
( #1728 ) exclude ephemeral models from freshness task
2019-09-10 11:15:58 -04:00
Jacob Beck
5c6f4ff637
disable dbt cache logging
2019-09-10 09:04:31 -06:00
Jacob Beck
693574212d
Merge pull request #1727 from fishtown-analytics/merge-0.14.1-into-lma
...
Merge dev/0.14.1 into dev/louisa-may-alcott
2019-09-05 15:24:17 -06:00
Jacob Beck
cd0e119737
re-order update_parsed_node so it makes logical sense
...
database -> schema -> alias instead of schema -> alias -> database
2019-09-05 14:52:40 -06:00
Jacob Beck
64193b5cc9
pr feedback
2019-09-05 14:49:24 -06:00
Jacob Beck
4a4fdb1452
Merge branch 'dev/0.14.1' into dev/louisa-may-alcott
2019-09-05 14:06:28 -06:00
Drew Banin
23bfc67d62
Merge pull request #1726 from fishtown-analytics/fix/0.14.2-docs-updates
...
(#1724 ) bump docs site for 0.14.2
2019-09-05 16:04:05 -04:00
Drew Banin
e0f725363c
( #1724 ) bump docs site for 0.14.2
2019-09-05 14:36:39 -04:00
Drew Banin
1dd4187cd0
Merge branch '0.14.latest'
2019-09-05 14:32:23 -04:00
Drew Banin
19f052e016
Merge pull request #1719 from fishtown-analytics/fix/dont-rerender-in-statements
...
(#1717 ) fix for re-rendering in statement calls
2019-09-04 09:22:33 -04:00
Connor McArthur
141bdd6f96
Bump version: 0.14.1rc2 → 0.14.1
2019-09-03 19:52:54 -04:00
Drew Banin
d1a49d3128
( #1717 ) fix for re-rendering in statement calls
2019-09-03 18:55:10 -04:00
Jacob Beck
48bd3cac39
Merge pull request #1720 from fishtown-analytics/refactor/rpc-pre-async
...
refactor dbt rpc in prep for async
2019-09-03 15:01:30 -06:00
Jacob Beck
db3c58ddb1
point dbt to a hologram v0.0.3, which handles List/List[str] encoding and decoding
2019-09-03 14:26:05 -06:00
Jacob Beck
279e4d5976
oops, we never use these (broken) methods
2019-09-03 09:20:40 -06:00
Jacob Beck
15840bd1c7
refactor dbt rpc in prep for async
...
Split rpc.py -> rpc/...
Move async entries in node_runners.py -> rpc/node_runners.py
Move RemoteCallable -> rpc/task.py
Move all RPC tasks -> task/remote.py
Clean up results for kill into a more coherent hologram type structure
Increased some test timeouts to improve some flaky tests around start-up compilation
Add many more type annotations for mypy
2019-09-03 08:49:27 -06:00
Jacob Beck
5c3874a392
Merge pull request #1715 from fishtown-analytics/feature/logbook-structured-rpc-logging
...
Feature/logbook structured rpc logging (#1703 , #1704 )
2019-08-30 12:35:46 -06:00
Jacob Beck
43c5c011d8
PR feedback
2019-08-30 12:09:19 -06:00
Jacob Beck
fb12595bdd
fix missing exception export
2019-08-30 11:36:24 -06:00
Jacob Beck
3163f090dd
Allow an initial disable to prevent a path from ever being set
2019-08-30 11:10:56 -06:00
Jacob Beck
e47b77d50a
refactor log management a bit to better support hooking into it
2019-08-30 10:56:19 -06:00
Jacob Beck
544dbfd746
Replace logging with logbook
...
Log rpc as json, preserve existing log behavior
Force werkzeug logs into logbook handling
Make rpc a package and set up mypy checking on it
fix a lot of logging-related bugs
Provide a log manager to toggle various log functionalities
add disabled flag to file handler for "dbt debug" and friends
Test changes to support logbook
lots of fiddly process/thread handling
2019-08-30 08:39:29 -06:00
Drew Banin
54f548eec8
Update CHANGELOG.md
2019-08-29 17:01:23 -04:00
Connor McArthur
786791670e
Bump version: 0.14.1rc1 → 0.14.1rc2
2019-08-28 18:08:14 -04:00
Drew Banin
ce58da82c5
Merge pull request #1709 from fishtown-analytics/fix/simplify-snapshot-check-logic
...
make snapshot check strategy simpler and still correct
2019-08-28 15:39:50 -04:00
Drew Banin
3437b0f2b9
Update CHANGELOG.md
2019-08-28 14:43:21 -04:00
Drew Banin
0970285956
make snapshot check strategy simpler and still correct
2019-08-28 11:30:27 -04:00
Drew Banin
c0028587dd
Merge pull request #1701 from fishtown-analytics/feature/expose-config-in-generation-macros
...
add configs to ParsedNodes before generating a schema/alias
2019-08-27 14:33:35 -04:00
Drew Banin
7fafa2adeb
Update CHANGELOG.md
2019-08-27 13:10:35 -04:00
Jacob Beck
513a3050c7
tests
2019-08-27 10:57:59 -06:00
Jacob Beck
57a2aae83f
log compiles to an array of log messages, disply it in status
2019-08-27 10:57:51 -06:00
Jacob Beck
c621e7d381
Fix some connection-related bugs
...
- fix a bug where failed connections caused an AttributeError
- fix an issue where the rpc server and its child processes secretly shared mutable state
2019-08-27 10:54:21 -06:00
Jacob Beck
fe48478993
Merge pull request #1699 from fishtown-analytics/feature/sighup-reload-manifest
...
reload RPC server manifest on SIGHUP (#1684 )
2019-08-27 07:05:23 -06:00
Drew Banin
d1cc5deaac
add tests for accessing config properties in generate macros
2019-08-26 22:54:17 -04:00
Drew Banin
952b1fc61b
add configs to ParsedNodes before generating a schema/alias
2019-08-26 22:10:51 -04:00
Jacob Beck
bd63aac8b3
PR feedback
2019-08-26 15:17:45 -06:00
Jacob Beck
9581f39186
Trigger rpc server reloads with sighup
...
From the sighup until completion, the sever enters an error state where only builtin commands work
- management of tasks should keep functioning properly
Add new builtin command 'status'
- returns 'pid', 'status' enum, 'timestamp'
- if the status is 'error', an 'error' field is populated
- just a Dict[str, Any] with, only 'message' for now
2019-08-26 10:01:47 -06:00
Jacob Beck
6649840b46
make mypy happy with deprecations
...
Requires git-only hologram branch
TODO: update hologram version + do another release
2019-08-26 09:53:04 -06:00
Drew Banin
1b03db6ab5
Update CHANGELOG.md
2019-08-26 09:55:45 -04:00
Connor McArthur
8a19ba4862
Bump version: 0.14.1a1 → 0.14.1rc1
2019-08-22 12:02:16 -04:00
Jacob Beck
eb8bce4e22
Merge branch 'dev/0.14.1' into dev/louisa-may-alcott
2019-08-21 16:04:19 -06:00
Drew Banin
f73d561557
Merge pull request #1678 from fishtown-analytics/0.14.1-changelog
...
0.14.1 changelog
2019-08-21 15:54:03 -04:00
Drew Banin
83003a7d8f
Update CHANGELOG.md
2019-08-21 15:50:48 -04:00
Jacob Beck
01534c1fc1
Merge pull request #1646 from fishtown-analytics/feature/know-about-files
...
Feature: Partial Parsing
2019-08-21 09:45:15 -07:00
Drew Banin
84a991520f
Merge pull request #1694 from fishtown-analytics/fix/connection-name-snowflake
...
Fix for errant method call in error handler
2019-08-21 09:40:33 -04:00
Drew Banin
89c4dbcdba
Fix for errant method call in error handler
2019-08-20 20:35:28 -04:00
Drew Banin
e7a24a2062
Merge pull request #1689 from bastienboutonnet/feature/snowflake_clustering
...
Implement Clustering for Snowflake
2019-08-20 12:27:22 -04:00
Drew Banin
0e897f7751
Merge pull request #1688 from mikaelene/project_cols_data_test
...
Updated injected_node.wrapped_sql for data tests
2019-08-20 12:26:14 -04:00
Drew Banin
ea898e25ad
Merge pull request #1690 from fishtown-analytics/fix/no-col-expansion-on-snowflake
...
(#1687 ) no-op column expansion on Snowflake + BQ
2019-08-20 11:54:40 -04:00
Mikael Ene
6d38226a1c
last try before I start over
2019-08-20 16:55:51 +02:00
Mikael Ene
b415e0eed3
last try before I start over
2019-08-20 16:54:02 +02:00
Bastien Boutonnet
90e8e75716
Apply suggestions from code review
...
Co-Authored-By: Drew Banin <drew@fishtownanalytics.com >
2019-08-20 16:27:06 +02:00
Drew Banin
5a74918e1e
pep8
2019-08-20 10:26:14 -04:00
Drew Banin
d8551a6c9c
use atomic column type alter statement on snowflake
2019-08-20 09:40:29 -04:00
Mikael Ene
51b546f50f
try again
2019-08-20 15:25:16 +02:00
Mikael Ene
4bc4c65795
fixed indention for flake and passed the local test. Cmon
2019-08-20 06:42:46 +02:00
Bastien Boutonnet
a0493b000e
add a check before clustering that cluster keys were provided
2019-08-19 21:21:24 +02:00
Bastien Boutonnet
7071c1d200
fix god aweful logical flow
2019-08-19 21:13:23 +02:00
Drew Banin
64ee763c99
Merge pull request #1686 from fishtown-analytics/fix/panoply
...
Fix for run and docs generate on Panoply Redshift
2019-08-19 13:04:01 -04:00
Drew Banin
e78d979d4c
Merge pull request #1650 from boxysean/add_postgres_ctas_adapter
...
Add Postgres CTAS adapter macro with support for unlogged parameter
2019-08-19 09:23:16 -04:00
mikael.ene
845529cc9d
made line shorter according to flake8
2019-08-19 12:01:38 +02:00
mikael.ene
c71b2dc312
made line shorter according to flake8
2019-08-19 11:52:03 +02:00
Sean McIntyre
d8775d1054
Add unlogged as a Postgres AdapterSpecificConfigs
2019-08-18 12:07:51 -04:00
Bastien Boutonnet
db2e564e7a
remove cluster key list join from if statement block
2019-08-17 21:21:04 +02:00
Bastien Boutonnet
7fc71272fd
apply patch from PR 1591
2019-08-17 15:46:29 +02:00
Mikael Ene
8011bc7b7a
changed the sql for data-tests for supporting sql server
2019-08-17 07:59:11 +02:00
Drew Banin
171fdf792e
( #1687 ) no-op column expansion on Snowflake + BQ
2019-08-16 15:18:00 -04:00
Drew Banin
7a01ba7b49
( #1479 ) fix for run and docs generate on Panoply Redshift
2019-08-16 10:51:52 -04:00
Connor McArthur
403d000a6f
Merge pull request #1680 from fishtown-analytics/pypi-hologram
...
hologram is on pypi
2019-08-13 17:01:13 -04:00
Connor McArthur
880b6666ef
hologram is on pypi
2019-08-13 16:50:14 -04:00
Drew Banin
2aee9eedad
Merge pull request #1670 from fishtown-analytics/fix/snowflake-unpickleable-datetime-timezone
...
fix for unpickleable datetime tzs set by snowflake
2019-08-13 13:17:04 -04:00
Connor McArthur
db203371fe
Bump version: 0.14.0 → 0.14.1a1
2019-08-12 20:18:30 -04:00
Drew Banin
50fa1baf96
fix for unpickleable datetime tzs set by snowflake
2019-08-12 13:46:46 -04:00
Drew Banin
da7c9501d2
Merge pull request #1673 from sjwhitworth/bq-priority
...
Support job priority in BigQuery
2019-08-09 19:27:56 -04:00
Stephen
be53b67e68
Add unit test for priority
2019-08-09 23:22:15 +01:00
Drew Banin
ffcaac5b4c
Merge pull request #1672 from sjwhitworth/fix-iternal-execption
...
Fix typo: "IternalException" -> "InternalException"
2019-08-09 14:46:10 -04:00
Stephen
65c3bf6d0f
Support job priority in BigQuery
2019-08-08 21:50:29 +01:00
Stephen
58976b46f9
Fix typo: "IternalException" -> "InternalException"
2019-08-08 20:25:14 +01:00
Jacob Beck
fe2a9fe097
Fix rebase issues
2019-08-06 13:20:50 -05:00
Drew Banin
3e3c69eaf9
Merge pull request #1663 from edmundyan/ey_create_schemas_lowered
...
Do case-insensitive schema comparisons to test for schema existance
2019-08-06 13:45:26 -04:00
Jacob Beck
bcb3df383f
PR feedback
...
fix compiled seeds to never be empty, like parsed seeds
target_database fixes
- make target_database/target_schema set node.database/schema
- use .database/.schema in the snapshot materialization
- allow target_database to not be set
- change a test to not include target_database to make sure we don't break it
- (carefully) rename misleading test
2019-08-06 07:45:44 -05:00
Jacob Beck
24c0179048
mypy rebase cleanup
2019-08-06 07:45:44 -05:00
Jacob Beck
e54661b5df
PR feedback
2019-08-06 07:45:44 -05:00
Jacob Beck
85cc8cd542
much better unit tests for the cache
2019-08-06 07:45:44 -05:00
Jacob Beck
9076806a9b
More windows nonsense
2019-08-06 07:45:44 -05:00
Jacob Beck
17378812a4
normcase everywhere, windows is terrible
2019-08-06 07:45:44 -05:00
Jacob Beck
0e97b53e70
PR feedback
...
fix windows tests
mypy + added more stubs for easy modules
remove __future__ imports from python2
gitignore
2019-08-06 07:45:42 -05:00
Jacob Beck
0caa90751b
invalidate the cache on profile/target changes
2019-08-06 07:44:15 -05:00
Jacob Beck
c38c8d539b
hide partial parsing behind a flag
2019-08-06 07:44:15 -05:00
Jacob Beck
23883303ff
Partial parsing
...
Refactor parsing
Store files in the manifest
some speed improvements
test fixes
Remove some old python 2 compatibility stuff
2019-08-06 07:44:13 -05:00
Jacob Beck
d08d2915e0
mypy quality of life fixes
2019-08-06 07:37:57 -05:00
Jacob Beck
1a618e7240
I can't live like this anymore, bumped to 0.15.0a1
2019-08-06 07:37:57 -05:00
Jacob Beck
60f66fc288
Make database/schema mandatory for all credentials
2019-08-06 07:37:57 -05:00
Jacob Beck
4faa633fb2
Merge pull request #1668 from fishtown-analytics/merge-0.14.1-into-lma
...
Merge 0.14.1 into louisa-may-alcott
2019-08-06 07:36:40 -05:00
Jacob Beck
8bb99547e3
Merge dev/0.14.1 into louisa-may-alcott
2019-08-06 02:42:47 -05:00
Edmund Yan
e867cfa4a2
Remove profile schema as we no longer run 'use schema'
2019-08-05 23:28:48 -04:00
Drew Banin
7901413a97
Merge pull request #1666 from fishtown-analytics/fix/snapshot-support-old-pg-versions
...
(#1665 ) Fix for casting error on old versions of postgres in snapshots
2019-08-05 22:50:49 -04:00
Drew Banin
b7e8670b43
( #1665 ) Fix for casting error on old versions of postgres in snapshot queries
2019-08-05 22:16:16 -04:00
Drew Banin
e2531edb02
Merge pull request #1662 from fishtown-analytics/template/bug-and-feature-templates
...
GitHub issue / PR touchups
2019-08-05 16:41:47 -04:00
Drew Banin
96913732e4
Merge pull request #1658 from fishtown-analytics/docs/0.14.1
...
Update CHANGELOG.md
2019-08-05 16:41:25 -04:00
Drew Banin
06a89446ab
Merge pull request #1664 from vitorbaptista/patch-1
...
Fix typo "paramteter" -> "parameter"
2019-08-05 16:31:21 -04:00
Vitor Baptista
ac128da45a
Fix typo "paramteter" -> "parameter"
2019-08-05 20:56:04 +01:00
Edmund Yan
265f6d3ce5
Do case-insensitive schema comparison before trying to create a schema
2019-08-05 14:12:36 -04:00
Drew Banin
388fd0bd00
Update bug report and feature request templates
2019-08-05 12:11:57 -04:00
Drew Banin
1c6945cb75
Update CHANGELOG.md
2019-08-05 11:23:58 -04:00
Drew Banin
772cb0d326
Update CHANGELOG.md
2019-08-05 10:50:28 -04:00
Drew Banin
b2f2e69377
Merge pull request #1540 from elexisvenator/fix/postgres-catalog
...
Change postgres `get_catalog` to not use `information_schema`
2019-08-04 19:29:30 -04:00
Drew Banin
377d5b7f58
Merge pull request #1657 from fishtown-analytics/bump/0.14.1-docs-site
...
bump docs site to 0.14.1
2019-08-04 19:25:22 -04:00
Drew Banin
b4da0686c8
bump docs site to 0.14.1
2019-08-04 19:20:38 -04:00
Drew Banin
f3baa69aad
Merge pull request #1656 from fishtown-analytics/fix/bad-profile-dir
...
(#1645 ) fix for errant cookie generation
2019-08-04 19:13:04 -04:00
Drew Banin
9527626ffe
Merge pull request #1609 from aminamos/dev/0.14.1
...
updated dbt.exceptions reference to exceptions in .sql files
2019-08-04 18:06:22 -04:00
Drew Banin
e13568117b
Merge pull request #1654 from fishtown-analytics/fix/summarize-warns
...
(#1597 ) summarize warnings at end of test invocations
2019-08-04 17:47:48 -04:00
Drew Banin
57c6b11d47
( #1645 ) fix for errant cookie generation
2019-08-03 14:14:04 -04:00
Drew Banin
800355ec2f
( #1597 ) summarize warnings at end of test invocations
2019-08-03 12:49:28 -04:00
Drew Banin
4478a89f28
Merge pull request #1647 from levimalott/fix/recover-from-failed-cleanup-rollbacks
...
Log, but allow, failures during cleanup rollbacks.
2019-08-02 16:25:08 -04:00
Jacob Beck
07aedc0e61
Merge pull request #1652 from fishtown-analytics/feature/dbt-rpc-run
...
dbt rpc tasks
2019-08-02 13:01:39 -06:00
Jacob Beck
115ef7dd97
Add some dbt project-level tasks to the RPC server
...
Add seed_project
- takes 'show' arg
Add run_project
- takes 'models' and 'exclude' args
Add compile_project
- takes 'models' and 'exclude' args
Add test_project
- takes models and exclude args
- takes 'data' and 'schema' args
Tests
2019-08-02 12:29:04 -06:00
Drew Banin
9e07912e1c
Merge pull request #1644 from fishtown-analytics/fix/seed-nonascii-chars
...
Fix for unicode chars in seed files
2019-08-02 13:31:41 -04:00
Jacob Beck
8fd768e46b
for agate, use the "Urb" mode on python 2, handle BOM fiddling
2019-08-01 12:41:51 -06:00
boxysean
dad3dcacfe
Add Postgres-specific CTAS adapter macro
2019-07-31 18:20:19 -04:00
Levi Malott
0927093303
Slim down the rollback failure log to appease flake8.
2019-07-31 16:22:29 -05:00
Levi Malott
3099119815
Log failed rollbacks to debug logs rather than to stdout.
2019-07-31 14:59:14 -05:00
Levi Malott
7a026c7e10
Log, but allow, failures during cleanup rollbacks.
...
In Postgres, rollbacks can fail if the transaction was killed
by the database. One common scenario is that the
`idle_in_transaction_session_timeout` is enabled.
If the
transaction was cancelled, the connection is left open
in `dbt`. `dbt` attempts to close that connection after issuing
a `ROLLBACK`. But it fails since the transaction was severed.
Since the cleanup is carried out in a `finally` statement, the
`psycopg2.InternalDatabaseError` is thrown and prevents the
test case results from ever being shown.
Changes here wrap the `ROLLBACK` in a try-catch such that
if there is an exception thrown, it is logged appropriately,
but ultimately proceeds.
2019-07-31 10:18:12 -05:00
Drew Banin
7177a6543b
Merge pull request #1638 from fishtown-analytics/fix/bq-repeated-record-execute
...
(#1626 ) fix for RPC error with BQ nested fields
2019-07-30 16:27:08 -04:00
Drew Banin
b2b0f78587
Merge pull request #1642 from fishtown-analytics/fix/ls-ephemeral-selection
...
exclude ephemeral addins in dbt ls command
2019-07-30 14:57:47 -04:00
Drew Banin
7001afbcbe
( #1632 ) fix for unicode chars in seed files
2019-07-30 14:33:32 -04:00
Drew Banin
3eb28198bd
remove unnecessary reordering of table cols
2019-07-30 12:47:02 -04:00
Drew Banin
81deb8d828
exclude ephemeral addins in dbt ls command
2019-07-30 12:39:11 -04:00
Drew Banin
e30ba80d6a
Merge pull request #1636 from fishtown-analytics/feature/test-freshness-disable
...
add test for nulled out freshness spec
2019-07-30 09:45:42 -04:00
Drew Banin
22d13ba881
serialize as json
2019-07-29 23:31:15 -04:00
Drew Banin
6cfbcf1ac8
( #1626 ) fix for RPC error with BQ nested fields
2019-07-29 22:35:16 -04:00
Drew Banin
21daca9faf
add test for nulled out freshness spec
2019-07-29 15:27:44 -04:00
Amin
142edcff38
Update core/dbt/exceptions.py
...
Co-Authored-By: Drew Banin <drew@fishtownanalytics.com >
2019-07-29 11:32:26 -07:00
aminamos
70d82ed48e
removed CODE from NotImplementedException class
2019-07-26 13:52:15 -07:00
Jacob Beck
52c9234621
Merge pull request #1629 from fishtown-analytics/feature/compile-tricks-flag
...
Add environment variables for macro debugging flags (#1628 )
2019-07-26 09:44:02 -06:00
Drew Banin
78d309551f
Merge pull request #1633 from fishtown-analytics/fix/source-column-description-interpolation
...
Fix for unrendered source column descriptions
2019-07-26 11:29:17 -04:00
Drew Banin
f91109570c
pep8
2019-07-26 09:19:31 -04:00
Drew Banin
291ef56bc7
( #1619 ) fix for unrendered source column descriptions
2019-07-25 22:25:04 -04:00
Drew Banin
b12484bb6f
Merge pull request #1614 from fishtown-analytics/fix/snapshot-check-cols-cycle
...
possible fix for re-used check cols on BQ
2019-07-24 12:56:38 -04:00
Jacob Beck
709ee2a0e8
Add environment variables for macro debugging flags
2019-07-23 17:26:53 -06:00
Drew Banin
8d4f2bd126
Merge pull request #1623 from fishtown-analytics/feature/set-snowflake-application-name
...
Set application name in snowflake connections
2019-07-22 13:21:59 -04:00
Drew Banin
b6e7351431
snapshot surrogate key whitespace control
2019-07-22 11:14:03 -04:00
Drew Banin
329145c13f
Merge branch 'dev/0.14.1' into fix/snapshot-check-cols-cycle
2019-07-21 15:26:44 -04:00
Drew Banin
35d1a7a1b5
add tests
2019-07-21 15:26:40 -04:00
Drew Banin
a2e801c2de
pep8
2019-07-21 13:40:43 -04:00
Drew Banin
e86c11e5de
set application name in snowflake connections
2019-07-20 15:59:36 -04:00
Jacob Beck
b0217ba299
Merge branch 'dev/0.14.1' into dev/louisa-may-alcott
2019-07-18 15:15:07 -04:00
Jacob Beck
62c3318fe5
Merge pull request #1610 from fishtown-analytics/feature/split-parsed-things
...
Split Parsed and Compiled nodes into subtypes (#1601 )
2019-07-18 12:52:23 -06:00
Jacob Beck
b9a3fe59c8
PR feedback
2019-07-18 11:12:48 -04:00
Jacob Beck
e46800f5b4
Merge pull request #1615 from fishtown-analytics/fix/linear-time-selection
...
Make node selection O(n)
2019-07-17 14:50:22 -06:00
Jacob Beck
0648737fc1
add the flat graph back in with caching, PR feedback
2019-07-17 15:30:19 -04:00
Jacob Beck
1a4daaba10
That was the last reference to to_flat_graph, so goodbye to all that
2019-07-17 12:52:35 -04:00
Jacob Beck
6be4ac044c
remove repated to_flat_graph() call
2019-07-17 12:46:44 -04:00
Jacob Beck
c0aabc7d0b
linear time wooo
2019-07-17 12:34:46 -04:00
Drew Banin
4df0bbd814
touchup var name and sql formatting
2019-07-16 23:43:03 -04:00
Drew Banin
5e6e746951
possible fix for re-used check cols on BQ
2019-07-16 23:24:19 -04:00
aminamos
a55a27acf6
removed extra line
2019-07-16 16:13:58 -07:00
aminamos
8046992e08
added string interpolation to raise_not_implemented
2019-07-16 14:28:31 -07:00
aminamos
de56e88a00
updated raise_not_implemented, commented on profile.py
2019-07-16 14:18:52 -07:00
Jacob Beck
72afd76e1a
Merge pull request #1589 from fishtown-analytics/feature/dataclasses-jsonschema-types
...
dataclasses-jsonschema types
2019-07-16 11:06:10 -06:00
Jacob Beck
12e53c732e
Split Parsed and Compiled nodes into subtypes
...
A few test changes to account for removed fields based on types
2019-07-16 13:04:02 -04:00
Jacob Beck
49f7cf8eca
Convert dbt to use dataclasses and hologram for representing things
...
Most of the things that previously used manually created jsonschemas
Split tests into their own node type
Change tests to reflect that tables require a freshness block
add a lot more debug-logging on exceptions
Make things that get passed to Var() tell it about their vars
finally make .empty a property
documentation resource type is now a property, not serialized
added a Mergeable helper mixin to perform simple merges
Convert some oneOf checks into if-else chains to get better errors
Add more tests
Use "Any" as value in type defs
- accept the warning from hologram for now, PR out to suppress it
set default values for enabled/materialized
Clean up the Parsed/Compiled type hierarchy
Allow generic snapshot definitions
remove the "graph" entry in the context
- This improves performance on large projects significantly
Update changelog to reflect removing graph
2019-07-16 12:00:31 -04:00
Jacob Beck
10be7bac2f
Merge branch 'dev/0.14.1' into dev/louisa-may-alcott
2019-07-16 11:42:42 -04:00
aminamos
fa6fb1b53d
updated dbt.exceptions reference to exceptions in .sql files
2019-07-15 19:14:22 -07:00
Drew Banin
5a1f0bdda5
Merge pull request #1608 from fishtown-analytics/fix/is-incremental-check-materialization
...
Make is_incremental live up to its name
2019-07-15 15:17:04 -04:00
Drew Banin
405748c744
make is_incremental live up to its name
2019-07-15 14:15:20 -04:00
Drew Banin
cc90b048af
Merge pull request #1605 from fishtown-analytics/dev/0.14.1-merge-wilt
...
Merge 0.14.latest into dev/0.14.1
2019-07-12 16:39:48 -04:00
Drew Banin
6886228992
Merge branch '0.14.latest' into dev/0.14.1-merge-wilt
2019-07-12 09:14:43 -04:00
Drew Banin
4569c905a5
Merge pull request #1596 from fishtown-analytics/fix/contrib-typos
...
Update CONTRIBUTING.md
2019-07-09 21:49:47 -04:00
Connor McArthur
453e81e895
Bump version: 0.14.0rc1 → 0.14.0
2019-07-09 21:44:06 -04:00
Drew Banin
3af8696761
Update CONTRIBUTING.md
2019-07-09 21:43:59 -04:00
Connor McArthur
399b33822a
update changelog date
2019-07-09 21:42:18 -04:00
Drew Banin
913a296cc4
Merge pull request #1583 from fishtown-analytics/update-contributing-guide
...
Update CONTRIBUTING.md
2019-07-09 17:59:45 -04:00
Drew Banin
bd55569703
Merge branch 'dev/wilt-chamberlain' into update-contributing-guide
2019-07-09 17:59:05 -04:00
Drew Banin
0e2d3f833d
Merge pull request #1593 from fishtown-analytics/fix/docs-links
...
Fix immediately-obvious links that will break
2019-07-09 17:49:57 -04:00
Drew Banin
3646969779
update changelog links
2019-07-09 15:39:07 -04:00
Claire Carroll
d5bfb9f6aa
Update docs link
2019-07-09 14:15:26 -04:00
Jacob Beck
5c05f709d8
Merge pull request #1594 from fishtown-analytics/feature/logging-again
...
Fix the test dockerfile to get the logfile's default encoding right
2019-07-09 10:20:26 -06:00
Jacob Beck
30a270b5f4
Fix the dockerfile to get the logfile's default encoding right
...
Fix a missing import
2019-07-09 08:46:08 -06:00
Jacob Beck
671836c47d
Merge branch 'dev/wilt-chamberlain' into dev/louisa-may-alcott
2019-07-09 08:23:25 -06:00
Drew Banin
56a2d9dc0a
Merge pull request #1590 from fishtown-analytics/fix/snapshot-check-cols-dupe-scd-id
...
[Wilt Chamberlain] fix for dupe check_cols values in snapshot strategy
2019-07-08 11:00:42 -04:00
Claire Carroll
e90c05c8f8
Fix immediately-obvious links that will break
2019-07-07 16:05:51 -04:00
Drew Banin
08d79cc324
( #1588 ) fix for dupe check_cols values in snapshot strategy
2019-07-05 21:14:54 -04:00
Drew Banin
990b0c93a5
Merge pull request #1587 from fishtown-analytics/fix/list-description
...
Fix the list subparser description
2019-07-03 17:04:45 -04:00
Claire Carroll
14d638c588
Fix the list subparser description
2019-07-03 16:34:47 -04:00
Jacob Beck
df8b12b2c0
Merge branch 'dev/wilt-chamberlain' into dev/louisa-may-alcott
2019-07-03 10:09:30 -06:00
Ben Edwards
2645667257
Fix table type
...
v = view
r, f, p = all are different forms of table
2019-07-03 07:32:54 +10:00
Ben Edwards
38c2d82c88
Cleaned up filtering to be in line with information_schema logic
2019-07-02 09:09:33 +10:00
Connor McArthur
986f5b7b4e
Bump version: 0.14.0a2 → 0.14.0rc1
2019-07-01 15:29:32 -04:00
Drew Banin
ef76c04ae8
Merge pull request #1585 from fishtown-analytics/fix/persist-docs-validation
...
add validation for extend dict fields that receive non-dicts
2019-07-01 13:18:52 -04:00
Drew Banin
d4d5393faa
add validation for extend dict fields that receive non-dicts
2019-07-01 11:45:46 -04:00
Drew Banin
9ffc4bf928
Update CONTRIBUTING.md
2019-06-28 12:26:26 -04:00
Jacob Beck
b9bfff19bb
Merge pull request #1581 from fishtown-analytics/feature/remove-archive-config
...
remove archive block support and "dbt snapshot-migrate" (#1580 )
2019-06-28 09:35:17 -04:00
Jacob Beck
51b6fd6f86
Merge pull request #1582 from fishtown-analytics/fix/set-io-encoding
...
set PYTHONIOENCODING on the dockerfile to suppress encoding errors
2019-06-27 23:15:14 -04:00
Jacob Beck
be765dc4e8
set PYTHONIOENCODING on the dockerfile to suppress encoding errors
2019-06-27 20:40:02 -06:00
Drew Banin
7febd9328d
Merge pull request #1542 from fishtown-analytics/changelog/0.14.0
...
Update CHANGELOG.md
2019-06-27 22:25:28 -04:00
Drew Banin
33a80fca5a
Update CHANGELOG.md
2019-06-27 22:24:51 -04:00
Jacob Beck
b0c6233b44
remove archive block support and "dbt snapshot-migrate"
2019-06-27 20:14:43 -06:00
Jacob Beck
f7158b233f
Merge branch 'dev/wilt-chamberlain' into dev/louisa-may-alcott
2019-06-27 15:47:12 -06:00
Jacob Beck
f368820b7e
Merge pull request #1578 from heisencoder/fix/test-cleanup
...
Fix/test cleanup
2019-06-27 12:06:26 -04:00
Matt Ball
2f1dbc2dae
change assert{Not}Equals to assert{Not}Equal in integration tests
...
This is to remove deprecation warnings during testing.
Linux commands (with globstar shell option enable):
$ cd test
$ sed -i 's/self.assertEquals/self.assertEqual/g' **/*.py
$ sed -i 's/self.assertNotEquals/self.assertNotEqual/g' **/*.py
2019-06-27 09:10:57 -06:00
Matt Ball
3ad30217c4
remove DeprecationWarnings from test/unit files
...
We're getting a bunch of deprecation warnings when running the unit
tests due to using statements like assertEquals instead of assertEqual.
This change removes these warnings.
Change completed via these Linux commands:
$ cd test/unit
$ sed -i 's/self.assertEquals/self.assertEqual/g' test_*.py
$ sed -i 's/self.assertNotEquals/self.assertNotEqual/g' test_*.py
2019-06-27 08:36:11 -06:00
Matt Ball
4a10f2cb37
update test_context.py to use a local import of mock_adapter
...
This is consistent with the way that unit tests import utils.py and
also fixes an import issue with our test environment.
2019-06-27 08:28:02 -06:00
Drew Banin
f3948295e9
Update CHANGELOG.md
2019-06-26 19:35:05 -04:00
Drew Banin
42fb12027c
Merge pull request #1575 from fishtown-analytics/tracking/adapter-type-and-rpc-requests
...
(#1574 ) track adapter type and rpc requests
2019-06-26 19:24:28 -04:00
Drew Banin
91124d2d4f
( #1574 ) track adapter type and rpc requests
2019-06-26 17:55:37 -04:00
Jacob Beck
19fe9119cb
Merge branch 'dev/wilt-chamberlain' into dev/louisa-may-alcott
2019-06-26 15:54:01 -06:00
Drew Banin
2e2ce9a57a
Update CHANGELOG.md
2019-06-26 17:52:59 -04:00
Drew Banin
fdcb395739
Merge pull request #1571 from josegalarza/dev/wilt-chamberlain
...
Add ExternalTable relation type, update Snowflake adapter (issue #1505 )
2019-06-26 11:24:32 -04:00
Drew Banin
7d1fed2eb9
Merge pull request #1572 from cclauss/patch-1
...
Undefined name: import shutil for line 60
2019-06-26 09:47:04 -04:00
cclauss
462a1516d2
Undefined name: import shutil for line 60
2019-06-26 13:24:12 +02:00
Jose
654f70d901
Add ExternalTable relation type, update Snowflake adapter (issue #1505 )
2019-06-26 12:11:41 +10:00
Jacob Beck
b09b1da1f4
Merge pull request #1568 from fishtown-analytics/feature/remove-agate-from-nodes
...
Remove agate from nodes
2019-06-25 14:33:24 -04:00
Jacob Beck
8ca49cb7af
lets not allow people to load arbitrary file paths, huh
2019-06-25 14:04:25 -04:00
Jacob Beck
e4d3942f36
set name back to agate_table in the sql
2019-06-25 12:48:29 -04:00
Jacob Beck
dd2f673083
Tests
...
make sure to run --show for every database
Fix the exit tests since we now fail at runtime, not compile time
2019-06-25 12:41:48 -04:00
Jacob Beck
43df84ae8f
plumb the loaded agate table through the results for --show
2019-06-25 12:21:12 -04:00
Jacob Beck
f211d4c8ec
Load agate tables at seed materialization time
2019-06-25 09:46:21 -04:00
Connor McArthur
31f20348c9
Bump version: 0.14.0a1 → 0.14.0a2
2019-06-25 09:45:09 -04:00
Jacob Beck
8c721ba561
Merge pull request #1535 from fishtown-analytics/feature/rm-rf-python2
...
Remove Python 2.7 support
2019-06-25 08:31:01 -04:00
Jacob Beck
01c0a5462b
PR feedback
2019-06-25 06:38:20 -04:00
Jacob Beck
f73575a8c1
python 3.7 docker/tox support
...
Dockerfile that supports 2.7,3.6,3.7
Add 2.7.16 so we can run old dbt tests on the same image as the current one (I hope!)
add 3.7 to tox.ini
2019-06-25 06:38:20 -04:00
Jacob Beck
c7385ec512
Remove python 2.7
...
remove tests for 2.7
remove all dbt.compat
remove six
assorted 2-removal related cleanups
do things that we could not do before due to py2
make super super()
classes always derive from obect in 3.x
Enum-ify enum-y things
azure pipelines -> python 3.7
mock is part of unittest now
update freezegun
2019-06-25 06:38:20 -04:00
Drew Banin
bd0876e2e6
Update CHANGELOG.md
2019-06-24 19:48:57 -04:00
Jacob Beck
4f3dc7629a
Merge branch 'dev/wilt-chamberlain' into dev/louisa-may-alcott
2019-06-21 15:49:08 -04:00
Jacob Beck
a47c09e5d2
Merge pull request #1565 from fishtown-analytics/feature/new-dockerfile
...
Pull in new dockerfile
2019-06-21 15:47:49 -04:00
Drew Banin
164468f990
Merge pull request #1562 from fishtown-analytics/feature/statement-sugar
...
add sugar over statements (run_query)
2019-06-21 13:40:19 -04:00
Jacob Beck
c56b631700
Pull in new dockerfile
2019-06-21 12:48:18 -04:00
Jacob Beck
84c487b577
Merge branch 'dev/wilt-chamberlain' into dev/louisa-may-alcott
2019-06-21 11:04:37 -04:00
Jacob Beck
2636969807
Merge pull request #1558 from fishtown-analytics/fix/created-relations-casing
...
Fix casing comparisons on dbt-created relations (#1555 )
2019-06-21 10:10:56 -04:00
Jacob Beck
1e17303b97
Merge pull request #1561 from fishtown-analytics/fix/revert-strict-undefined
...
Remove strict undefined + tests
2019-06-20 15:22:07 -06:00
Drew Banin
7fa8d891ef
Merge pull request #1560 from fishtown-analytics/fix/support-error-on-nondeterministic-merge-snowflake
...
add strategy switch for incremental merge behavior (snowflake)
2019-06-20 16:36:39 -04:00
Drew Banin
c029dfe3fa
add sugar over statements (run_query)
2019-06-20 16:33:31 -04:00
Drew Banin
c4ef120b74
Merge branch 'dev/wilt-chamberlain' into fix/support-error-on-nondeterministic-merge-snowflake
2019-06-20 14:56:13 -04:00
Drew Banin
8644ce1cb8
rm strategy alias; rename overwrite strategy
2019-06-20 14:52:32 -04:00
Jacob Beck
b0b3cdc21f
Remove strict undefined + tests
2019-06-20 14:48:07 -04:00
Drew Banin
e3d30d8a35
add strategy switch for incremental merge behavior (snowflake)
2019-06-20 14:04:24 -04:00
Jacob Beck
0ef9c189c0
Merge pull request #1550 from fishtown-analytics/fix/block-parser-ignore-more
...
clean up block parsing to make it dumber and more effective (#1547 )
2019-06-20 06:33:48 -06:00
Jacob Beck
18953536f1
PR feedback
2019-06-19 14:36:07 -06:00
Jacob Beck
dd02f33482
new test
2019-06-19 11:33:57 -06:00
Jacob Beck
3576839199
oops, fix bq/sf too
2019-06-19 11:33:49 -06:00
Drew Banin
4ed668ef93
Update CHANGELOG.md
2019-06-19 13:30:06 -04:00
Drew Banin
f8344469e1
Merge pull request #1549 from heisencoder/feature/add-project-dir-flag
...
add --project-dir flag to allow specifying project directory
2019-06-19 13:28:24 -04:00
Jacob Beck
1d94fb67da
Fix casing comparisons on dbt-created relations
...
When dbt creates a relation in the db, add a special flag
When checking a node name match:
- if that flag is present and quoting is disabled, do a lowercase compare
- otherwise remain case sensitive
2019-06-19 09:44:37 -06:00
Drew Banin
9ad85127e4
Merge pull request #1553 from fishtown-analytics/fix/include-stuff
...
Add macro subdirectories to redshift/postgres (#1552 )
2019-06-18 19:13:46 -04:00
Drew Banin
3845abeff8
Update CHANGELOG.md
2019-06-18 18:34:52 -04:00
Jacob Beck
f95c712f95
Merge pull request #1551 from fishtown-analytics/fix/package-without-version
...
require versions in package defs (#1546 )
2019-06-18 13:16:45 -06:00
Matt Ball
f5c3300304
remove tab character in base.py
2019-06-18 13:08:53 -06:00
Matt Ball
84fa83b4dd
add unit test for new --project-dir flag
...
See https://github.com/fishtown-analytics/dbt/issues/1544
2019-06-18 12:51:02 -06:00
Jacob Beck
927c37470a
a different message
2019-06-18 12:41:57 -06:00
Jacob Beck
b80fa53b2a
include macros/**/*.sql as well
2019-06-18 12:19:48 -06:00
Jacob Beck
cce5ae01f8
require versions
2019-06-18 11:21:06 -06:00
Jacob Beck
92ef783948
clean up block parsing to make it dumber and more effective
2019-06-18 10:45:21 -06:00
Matt Ball
2e7c1fd2cc
add --project-dir flag to allow specifying project directory
...
In particular, this is the directory that contains the dbt_project.yml
file and all the related project files and directories.
Without this flag, it is necessary to cd to the project directory before
running dbt.
See https://github.com/fishtown-analytics/dbt/issues/1544
2019-06-17 16:22:45 -06:00
Jacob Beck
85164b616e
Merge pull request #1543 from fishtown-analytics/fix/unquoted-strings-errors
...
Handle quotes in jinja block bodies (#1533 )
2019-06-14 12:03:18 -06:00
Jacob Beck
c67a1ac9f7
link to issue in test case comment
2019-06-14 11:37:24 -06:00
Jacob Beck
a2cae7df29
Block parsing now uses recursive descent...
2019-06-14 10:50:32 -06:00
Drew Banin
f44e3bc9d8
Update CHANGELOG.md
2019-06-14 09:44:24 -07:00
Ben Edwards
03bc58116c
Fix incorrectly named column table_type
2019-06-14 17:21:08 +10:00
Ben Edwards
9b88eb67a1
Change postgres get_catalog to not use information_schema
...
- `information_schema` in Postgres is not very performant due to the complex views used to create it
- use underlying `pg_catalog` tables/views instead
- returns the same rows/columns as the `information_schema` version
- order of rows is different, this is because there was only a partial sort on the `information_schema` version
- `column_type` will return different values to before
- some arrays were `ARRAY`, will now be `type[]`
- user-defined types were previously `USER_DEFINED`, now will be the name of the user-defined type <-- main point of this PR
- performance is 2-5x faster, depending on query caching
2019-06-14 13:51:10 +10:00
Connor McArthur
6e5fa7de3c
add changelog section for wilt
2019-06-13 17:16:21 -04:00
Jacob Beck
d327394057
Merge pull request #1537 from fishtown-analytics/fix/bigquery-fix-schema-404s
...
List 10k datasets instead of all (#1536 )
2019-06-13 14:32:24 -06:00
Drew Banin
b7c06941e3
Merge pull request #1539 from fishtown-analytics/docs/0.14.0-updates
...
(#1474 ) incorporate docs site updates
2019-06-13 10:42:44 -07:00
Drew Banin
57adfc8683
( #1474 ) incorporate docs site updates
2019-06-13 10:22:00 -07:00
Jacob Beck
e13d805197
list a ton of datasets instead of all to avoid pagination
2019-06-13 10:42:20 -06:00
Jacob Beck
f0635a0df4
Merge pull request #1514 from fishtown-analytics/fix/dbt-ls-selectors
...
Fix dbt ls selectors
2019-06-13 10:40:57 -06:00
Jacob Beck
24adb74498
Merge pull request #1526 from fishtown-analytics/fix/refs-in-run-operations
...
Add ref and source support to run-operation macros (#1507 )
2019-06-13 10:18:33 -06:00
Jacob Beck
16519b11aa
Merge pull request #1534 from fishtown-analytics/fix/deprecate-python-2
...
Emit a big deprecation message on python 2.x at the end of the run (#1531 )
2019-06-13 10:14:10 -06:00
Jacob Beck
0a666caa13
Emit a big deprecation warning on python 2.x at the end of the run
2019-06-13 08:44:40 -06:00
Jacob Beck
7525da70ef
Merge branch 'dev/wilt-chamberlain' into dev/louisa-may-alcott
2019-06-12 20:33:05 -06:00
Jacob Beck
5833acbc8c
Print fqns, not unique ids
...
FQNs for sources
Allow package names as an optional prefix for source: selectors
make "dbt ls" output more intuitive by including source:* by default
Fix the tests
2019-06-12 19:46:27 -06:00
Jacob Beck
e57c7b651d
Merge pull request #1519 from fishtown-analytics/feature/render-severity-configs
...
render severity configurations (#1512 ) (#1511 )
2019-06-12 19:40:31 -06:00
Jacob Beck
9a40395cdb
Add ref and source support to run-operation macros
...
Removed the "runtime" provider's generate_macro function
Added an "operation" provider that does similar but:
ref() does not check for itself in depends_on
source() does not try to append to sources
Macros assume that the rendered form the the ref exists in the database
Also changed `--macro` to a required positional argument
you can run macro_name with 'dbt run-operation macro_name --args ...'
Make providers objects, unify some ref processing stuff
Remove an empty method call on operations, add some comments
Raise a compiler error if you try to ref() an ephemeral model in your run-operation macro
2019-06-12 19:36:16 -06:00
Jacob Beck
03aecc8d0c
Merge pull request #1481 from fishtown-analytics/feature/archive-migration-script
...
archive table migration script
2019-06-12 19:01:59 -06:00
Jacob Beck
a554b383a2
Merge pull request #1520 from fishtown-analytics/feature/rename-models-to-select
...
for snapshots, make the argument "--select", not "--models" (#1517 )
2019-06-12 16:14:04 -06:00
Jacob Beck
a4be1e1dcb
when warning about undefined, handle the case where the node is none
2019-06-12 16:13:25 -06:00
Jacob Beck
7b498f4179
archive table migration script
...
Also migrate archive configs
PR feedback
archives are now snapshots
Make the script a subcommand
2019-06-12 15:45:43 -06:00
Jacob Beck
4a10c8dce7
fix bigquery test
2019-06-12 15:39:17 -06:00
Jacob Beck
0cc99c50a7
for snapshots, make the argument "--select", not "--models"
2019-06-12 15:39:12 -06:00
Jacob Beck
92fdf45f0c
Support lowercase test severity
2019-06-12 15:38:05 -06:00
Jacob Beck
f3cafae030
render severity configurations
2019-06-12 15:37:35 -06:00
Jacob Beck
12e5bf6036
Merge pull request #1522 from fishtown-analytics/fix/check-cols-null
...
handle null check-cols (#1516 )
2019-06-12 15:32:40 -06:00
Jacob Beck
f0ab957edb
Merge pull request #1521 from fishtown-analytics/fix/quote-columns-in-snapshots
...
Snapshots: Quote columns, fix print output, fix tests (#1510 )
2019-06-12 15:32:11 -06:00
Jacob Beck
4308b28aa6
Merge pull request #1527 from fishtown-analytics/feature/parallel-integration-tests
...
parallel tests
2019-06-12 15:07:40 -06:00
Jacob Beck
f79619ec0a
parallelize the tests
...
Set -n4 in ci, as -n auto chose 32 and caused problems
python 2.x and flaky/pytest-xdist do not get along, so py3 only
Mark some rpc tests as flaky, especially with parallelism
- socket contention is a problem
Remove coverage stuff
Coallate and upload the log files we generate
Added special logging for a particular rare CI error case
Set an "initial root" at import time
chdir into it at setUp time as well as tearDown time
I think py2 might work now?
2019-06-12 14:37:54 -06:00
Jacob Beck
5dd147123a
HACK to get docs tests happy, need to settle on rules for what path emits wrt symlinks
2019-06-12 14:37:34 -06:00
Jacob Beck
63d6ab2006
make integration tests run in their own directories
...
Make tempdir, write profile + project there, use it as the profile directory
fixed 015 tests to turn that behavior off and do custom things
Migrate all the tests
Make run_dbt_and_check more like run_dbt
Fix windows ls tests
2019-06-12 14:37:07 -06:00
Jacob Beck
1951e0f5e8
Merge pull request #1532 from fishtown-analytics/build/pr-fixup-1285
...
merged and updated PR for #1031
2019-06-12 14:29:13 -06:00
Jacob Beck
fb69b89ef9
Merge pull request #1509 from heisencoder/feature/upgrade-to-networkx-2
...
Feature/upgrade to networkx 2
2019-06-12 12:07:25 -06:00
Jacob Beck
2d84dd4fbd
Merge branch 'dev/wilt-chamberlain' into build/pr-fixup-1285
2019-06-12 11:43:08 -06:00
Jacob Beck
d760229abc
Merge pull request #1476 from buremba/fix/pg-schema
...
Pass schema in credentials to Postgresql
2019-06-12 11:38:35 -06:00
Jacob Beck
ffb38a21e3
redshift tests
2019-06-12 10:45:03 -06:00
Matt Ball
85a2f48c80
update test_linker.py to handle a temp file using try/finally
...
This is based on the recommendation from @beckjake:
b8d5a341f4 (discussion_r292032021)
2019-06-12 10:33:34 -06:00
Jacob Beck
9f208f711e
Merge pull request #1515 from tbescherer/project-inner-loop
...
move target_model vars inside loop to avoid reuse on subsequent refs
2019-06-10 08:29:15 -06:00
Jacob Beck
ea2637395c
Merge pull request #1525 from fishtown-analytics/fix/increase-tracking-timeout
...
increase tracking timeout 2s -> 5s (#1499 )
2019-06-10 08:07:27 -06:00
Jacob Beck
0ca6026124
increase tracking timeout 2s -> 5s
2019-06-10 07:13:46 -06:00
Matt Ball
b8d5a341f4
update test_graph.py to handle different sequence type in networkx 2.x
2019-06-07 14:56:08 -06:00
Matt Ball
014a8f9222
Merge branch 'dev/wilt-chamberlain' of https://github.com/fishtown-analytics/dbt into feature/upgrade-to-networkx-2
2019-06-07 14:05:21 -06:00
Jacob Beck
9599b3f584
handle null check-cols
2019-06-07 14:00:14 -06:00
tom.bescherer
ca31b79cc0
add a newline for readability
2019-06-07 11:57:19 -04:00
Jacob Beck
675e858050
Quote columns, fix print output, fix tests
2019-06-07 09:19:36 -06:00
Jacob Beck
c04517bf04
Merge pull request #1506 from fishtown-analytics/feature/rename-archives-to-snapshots
...
Rename archives to snapshots (#1497 )
2019-06-07 07:17:47 -06:00
tom.bescherer
eb12ef1dcd
move target_model vars inside loop to avoid reuse on subsequent refs
2019-06-06 16:40:32 -04:00
Jacob Beck
03f50f560b
PR feedback, fix "usage" line
2019-06-06 14:35:22 -06:00
Jacob Beck
c1387c5692
Merge pull request #1513 from fishtown-analytics/fix/only-parse-docs-blocks
...
Only parse docs blocks contents when reading md files (#988 )
2019-06-06 13:53:15 -06:00
Jacob Beck
a4a9221d95
add a missing test file
2019-06-06 12:04:53 -06:00
Matt Ball
7ed0036af0
update networkx dependency to allow version 2.x
...
https://github.com/fishtown-analytics/dbt/issues/1496
2019-06-06 11:03:14 -06:00
Jacob Beck
1489393489
documentation -> docs
2019-06-06 10:59:29 -06:00
Matt Ball
c4939368ae
update linker.py to be compatible with networkx-2.x
...
Also, update test_linker.py to run under Python 2 unit tests and add
test coverage for the modified functions.
linker.py should now be compatible with both networkx-1 and -2.
This addresses https://github.com/fishtown-analytics/dbt/issues/1496
This change was tested via unit tests, but was not tested via
integration tests.
As a follow-up change, core/setup.py should have its networkx dependency
updated to allow version 2.x.
2019-06-06 10:31:36 -06:00
Jacob Beck
ab59ebe4f2
Fix docs blocks parsing issues
...
Rename documentation node type to docs so we can filter on it (is this breaking?)
Fix block extractor bug with macros/docs that contain quotes
Fix block extractor bug with expressions
2019-06-06 09:32:50 -06:00
Jacob Beck
f3701ab837
archives -> snapshots, except legacy stuff
2019-06-05 09:43:11 -06:00
Jacob Beck
f48f78fc58
rename a couple things, this will not work atm
2019-06-05 07:45:55 -06:00
Jacob Beck
ddb1785698
Merge pull request #1473 from fishtown-analytics/fix/windows-rpc
...
Fix windows rpc
2019-06-04 15:50:46 -06:00
Jacob Beck
963b0e23ee
remove drop_existing flag + its deprecation warning, make flags update themselves, reset flags from args on windows processes
2019-06-04 12:01:07 -06:00
Jacob Beck
2a9ae83270
Re-enable windows tests
...
Disable kill tests
Don't expect logs on timed-out windows tests (windows is slow!)
2019-06-04 11:59:33 -06:00
Jacob Beck
788507e046
Get windows up and running for RPC
...
Refactor process bootstrapping to a function
- avoid trying to pickle "RequestTaskHandler"s on windows
Move user config consequences out of main
- We need this for RPC stuff too
Reset profile values and plugins on windows process start
Disable "kill" command on Windows
2019-06-04 11:59:33 -06:00
Jacob Beck
60001ad6b4
Merge pull request #1483 from fishtown-analytics/fix/swap-alias-generator-args
...
flip around generate_alias_name args, add node to generate_schema_name args
2019-06-04 09:32:20 -06:00
Jacob Beck
6c9d5c7370
Merge pull request #1500 from fishtown-analytics/feature/remove-archive-blocks
...
remove archive configs
2019-06-04 09:32:08 -06:00
Jacob Beck
1c3a02b2c8
PR feedback
2019-06-04 07:20:55 -06:00
Jacob Beck
28dc10ed98
PR feedback
2019-06-03 16:32:20 -06:00
Jacob Beck
0d49295b94
tests, my ancient nemesis
2019-06-03 16:32:14 -06:00
Jacob Beck
679784735e
Swap aliases ordering and add node parameter to generate_schema_name
...
Fix many tests
Support single-arg generate_schema_name macros
Add repeat flag to warn_or_error to suppress duplicate warnings
Add a warning if a user's macro does not take a second argument
2019-06-03 16:32:14 -06:00
Drew Banin
99f62e850f
fix tests
2019-06-03 16:32:14 -06:00
Drew Banin
001b9abce9
flip around generate_alias_name args
2019-06-03 16:32:14 -06:00
Jacob Beck
248ca3ff76
fix more tests
2019-06-03 16:22:50 -06:00
Jacob Beck
704ee58846
fix the exit code tests too
2019-06-03 16:17:21 -06:00
Jacob Beck
f26948dde2
remove archive blocks
2019-06-03 15:57:32 -06:00
Drew Banin
3cac2d3ab7
Merge pull request #1478 from fishtown-analytics/feature/archive-blocks-as-regex-materialization
...
Use merge pattern for Archival queries
2019-06-03 11:50:18 -04:00
Drew Banin
82793a02d3
fix for tests in different logical databases
2019-06-01 11:40:14 -04:00
Jacob Beck
434eb94730
Merge branch 'dev/wilt-chamberlain' into dev/louisa-may-alcott
2019-05-30 13:26:00 -06:00
Jacob Beck
73b70622a1
Merge pull request #1490 from fishtown-analytics/refactor/node-selector-separated
...
Refactor node selection
2019-05-30 12:59:21 -06:00
Jacob Beck
00cbe3ec3b
Merge pull request #1494 from fishtown-analytics/fix/render-descriptions-properly
...
Render source_description fields in sources (#1484 )
2019-05-30 12:58:48 -06:00
Drew Banin
8ecdab817a
fix for pg tests
2019-05-30 13:14:27 -04:00
Drew Banin
94ae9fd4a7
fix test
2019-05-30 13:08:27 -04:00
Jacob Beck
75c8f32186
Render source_description fields in sources, fix tests to make sure we actually do that...
2019-05-30 10:54:47 -06:00
Drew Banin
69621fe6f9
cleanup tests
2019-05-30 12:21:42 -04:00
Drew Banin
81f4c1bd7c
cleanup merge
2019-05-30 12:18:58 -04:00
Drew Banin
bb7cfb7dc2
Merge branch 'dev/wilt-chamberlain' into feature/archive-blocks-as-regex-materialization
2019-05-30 12:17:24 -04:00
Drew Banin
b98ea32add
code review
2019-05-30 12:11:08 -04:00
Jacob Beck
2b89c5cd9a
shut up assertEquals warnings
2019-05-30 10:04:17 -06:00
Jacob Beck
74e5b8bdff
factor node selection into components that know about the manifest vs the graph
2019-05-30 10:04:17 -06:00
Jacob Beck
17157f2973
Merge pull request #1489 from fishtown-analytics/fix/redshift-concurrent-tests
...
fix concurrent transactions test
2019-05-30 09:58:01 -06:00
Drew Banin
478b17a4dc
fixups
2019-05-30 11:12:22 -04:00
Jacob Beck
f14225f7e4
Merge pull request #1491 from fishtown-analytics/feature/hub-site-retries
...
add a retry + sleep loop to registry calls (#1451 )
2019-05-30 07:31:40 -06:00
Drew Banin
8a8f7a9929
Merge branch 'dev/wilt-chamberlain' into build/pr-fixup-1285
2019-05-30 09:07:11 -04:00
Drew Banin
7d490d4886
Implement archival using a merge abstraction
2019-05-29 21:41:10 -04:00
Jacob Beck
af13b2c745
add a retry + sleep loop to registry calls
2019-05-29 15:29:53 -06:00
Jacob Beck
a164d83dad
in the concurrent transactions test, use a completely separate adapter for our goofy sql running
2019-05-29 13:08:33 -06:00
Jacob Beck
d10e340823
Merge pull request #1482 from fishtown-analytics/fix/archives-no-overwrite
...
detect duplicate archive and node names (#1480 )
2019-05-29 06:15:42 -06:00
Jacob Beck
e7bb9d14b2
Merge pull request #1475 from fishtown-analytics/fix/faster-snowflake-tests
...
Try to make snowflake tests a bit more performant (#1433 )
2019-05-29 06:13:26 -06:00
Burak Emre Kabakcı
4e7c096c34
Fix Code Style
2019-05-28 12:59:58 +03:00
Burak Emre Kabakcı
0f5ce12dad
Update .gitignore
2019-05-27 15:09:41 +03:00
Burak Emre Kabakcı
2f4e92a728
Fix linting issues
2019-05-27 15:08:14 +03:00
Jacob Beck
b047ed82b6
detect duplicate archive and node names
2019-05-24 12:47:51 -06:00
Burak Emre Kabakcı
82f165625f
Use get method for extracting search_path
2019-05-24 11:55:39 +03:00
Burak Emre Kabakcı
2f5aa3bd0e
Remove redundant init
2019-05-22 14:24:03 +03:00
Burak Emre Kabakcı
63ef8e3f17
Rename config to search_path
2019-05-22 14:14:05 +03:00
Burak Emre Kabakcı
f1eaeb4ed2
Fix typo in Redshift tests
2019-05-21 19:48:32 +03:00
Burak Emre Kabakcı
a2ffe8e938
Add tests & fix failing tests
2019-05-21 19:36:29 +03:00
Burak Emre Kabakcı
355d2ad6fc
Pass schema in credentials to Postgresql
2019-05-21 19:09:06 +03:00
Jacob Beck
f089b4b077
try to make snowflake assertManyTablesEqual faster
...
use "show columns" instead of selecting from the information schema
calculate numbers of rows while we calculate mismatched rows
use "show objects" instead of selecting from the information schema for get_models_in_schema
2019-05-21 09:30:32 -06:00
Jacob Beck
73607b85b7
Merge pull request #1465 from fishtown-analytics/dev/merge-in-0.13.1
...
merge 0.13.1
2019-05-16 07:23:55 -06:00
Drew Banin
7b022f3afa
Merge pull request #1459 from fishtown-analytics/feature/printer-width-tests
...
Make printer width configurable
2019-05-15 17:49:50 -04:00
Drew Banin
0a2e4f761b
Merge pull request #1409 from bastienboutonnet/snowflake_create_or_replace
...
Snowflake create or replace
2019-05-15 17:48:25 -04:00
Jacob Beck
ddd73cd73b
Merge branch '0.13.latest' into dev/wilt-chamberlain
2019-05-15 12:44:44 -06:00
Kriselle Sanchez
efdb837a50
Make printer width configurable
2019-05-13 14:52:20 -04:00
Drew Banin
90abc2d2f3
( closes #1455 ) Qualify Snowflake temp tables with a database and schema
2019-05-13 14:12:30 -04:00
Connor McArthur
f60938aab0
Bump version: 0.13.1a2 → 0.13.1
2019-05-13 11:58:03 -04:00
Connor McArthur
af7c752fc6
set release date
2019-05-13 11:57:29 -04:00
Connor McArthur
a80989952a
Merge pull request #1392 from fishtown-analytics/dev/0.13.1
...
dbt 0.13.1
2019-05-13 11:56:37 -04:00
Drew Banin
8d74550609
fix tests
2019-05-10 19:12:38 -04:00
Jacob Beck
d5774b3da4
Merge pull request #1452 from fishtown-analytics/feature/strip-seed-bom
...
Handle seeds with utf-8 BOM (#1177 )
2019-05-10 10:40:53 -06:00
Jacob Beck
7f7002f36c
Merge pull request #1454 from fishtown-analytics/feature/increment-version
...
Bump version: 0.13.0 → 0.14.0a1
2019-05-10 10:40:41 -06:00
Drew Banin
b62ba4a985
py2 compat
2019-05-10 10:52:57 -04:00
Drew Banin
2b3370887e
add missing import
2019-05-10 10:19:27 -04:00
Jacob Beck
26427d2af0
PR feedback
2019-05-10 07:50:46 -06:00
Jacob Beck
d502b33ef4
Merge pull request #1453 from fishtown-analytics/feature/warn-on-unpinned-packages
...
Warn on unpinned git packages (#1446 )
2019-05-10 07:01:28 -06:00
Jacob Beck
210cf43574
Merge pull request #1440 from fishtown-analytics/feature/output-run-hooks
...
Output run hooks (#696 )
2019-05-09 21:06:37 -06:00
Jacob Beck
d74e37d4ea
Always specify encodings to open()
...
Fixes some neat windows behavior where the default code page can be cp1252
Add BOM unit test
2019-05-09 21:04:52 -06:00
Jacob Beck
ea8825996d
PR feedback
2019-05-09 20:57:56 -06:00
Jacob Beck
336368195e
Bump version: 0.13.0 → 0.14.0a1
2019-05-09 20:21:24 -06:00
Jacob Beck
70206b1635
warn on unpinned dependencies, unless warn-unpinned: False is set in packages.yml
2019-05-09 20:09:20 -06:00
Jacob Beck
191ae61b02
python 2
2019-05-09 19:48:26 -06:00
Jacob Beck
f6bf8d912a
PR feedback
...
Order hooks in a deterministic way:
- in the root project, hooks in index order
- foreach dependency project in alphabetical order, hooks in index order
Since these hooks always have a valid "index", no need to devise a default
Add some tests
2019-05-09 19:16:53 -06:00
Jacob Beck
30b7407597
if the first byte of a csv seed is a BOM, strip it before passing it to agate
2019-05-09 17:56:05 -06:00
Jacob Beck
58bf73d4bf
Update core/dbt/task/run.py
...
Co-Authored-By: Drew Banin <drew@fishtownanalytics.com >
2019-05-09 14:30:30 -06:00
Drew Banin
ec61073560
update plugin readmes
2019-05-09 15:58:44 -04:00
Drew Banin
b0f81edf96
0.13.1a2 bump, include README.md in pypi description
2019-05-09 15:14:49 -04:00
Jacob Beck
cc8ef47747
Merge pull request #1436 from fishtown-analytics/feature/dbt-ls
...
dbt ls command (#467 )
2019-05-09 12:55:06 -06:00
Jacob Beck
5d05bf0aba
PR feedback
2019-05-09 12:54:38 -06:00
Drew Banin
1d18a54b5e
Update CHANGELOG.md
2019-05-09 14:52:17 -04:00
Drew Banin
a4605ec844
Merge pull request #1450 from fishtown-analytics/0-13-1-docs
...
bump docs for 0.13.1
2019-05-09 14:46:03 -04:00
Drew Banin
9f58400ba8
bump docs for 0.13.1
2019-05-09 14:44:42 -04:00
Jacob Beck
ec8277b0e4
add color to hooks, give hooks names
2019-05-09 10:53:46 -06:00
Jacob Beck
8c7763acf6
Merge pull request #1445 from fishtown-analytics/fix/anonymous-tracking-no-hangs
...
Improve tracking failure handling (#1063 )
2019-05-09 10:47:06 -06:00
Jacob Beck
8e426e60c9
remove completed TODO
2019-05-09 10:40:46 -06:00
Jacob Beck
3a7f931a3a
Merge pull request #1428 from fishtown-analytics/fix/strip-url-trailing-dotgit
...
On mostly-duplicate git urls, pick whichever came first (#1084 )
2019-05-08 19:33:07 -06:00
Drew Banin
314ca6c361
Update core/dbt/tracking.py
...
Co-Authored-By: beckjake <beckjake@users.noreply.github.com >
2019-05-08 12:40:33 -06:00
Jacob Beck
12f0887d28
PR feedback
...
remove incorrect comment
fix an issue where we looked deps up by the wrong key
add a test
2019-05-08 10:24:24 -06:00
Jacob Beck
a986ae247d
Improve tracking failure handling
...
Add a 1s timeout to tracking http calls
Add an on-failure handler that disables tracking after the first failure
2019-05-08 09:43:32 -06:00
Jacob Beck
9507669b42
PR feedback
...
Truncate long run-hooks, replacing newlines with spaces
Include run-hook status/timing in output
Clean up run-hook execution a bit
2019-05-07 09:53:37 -06:00
Jacob Beck
715155a1e9
Merge pull request #1441 from fishtown-analytics/feature/dynamic-target-paths
...
Add modules and tracking info to the configuration parsing context (#1320 )
2019-05-07 08:00:11 -06:00
Jacob Beck
32c5679039
PR Feedback
...
Fixed error logging to display errors in dbt ls
Add models flag
Make all of models, select, exclude have a metavar of 'SELECTOR' for -h
2019-05-07 07:57:06 -06:00
Jacob Beck
89d211b061
Merge pull request #1442 from fishtown-analytics/fix/no-source-links
...
Only link schemas with executable things in them
2019-05-06 14:27:18 -06:00
Jacob Beck
f938fd4540
fix a test bug
...
dbt doing two things in the same second incorrectly failed the test
2019-05-06 12:24:40 -06:00
Jacob Beck
61e4fbf152
add a test
2019-05-06 11:27:16 -06:00
Jacob Beck
fda38e7cbb
Only link schemas with executable things in them
2019-05-06 10:46:10 -06:00
Jacob Beck
c0a3b02fb4
Add modules and tracking info to the config parsing context
...
Add a test to make sure setting the target path works properly
2019-05-03 10:45:09 -06:00
Jacob Beck
80482aae34
add on-run-start/on-run-end hook logging
2019-05-03 08:00:08 -06:00
Jacob Beck
c19085862a
suppress some cache logger noise
2019-05-03 07:59:33 -06:00
Connor McArthur
9672d55c1e
Bump version: 0.13.0 → 0.13.1a1
2019-05-03 09:22:37 -04:00
Jacob Beck
e043643a54
Add a new ListTask, and 'dbt list'/'dbt ls'
...
All tasks now have a 'pre_init_hook' classmethod, called by main
- runs after args are parsed, before anything else
2019-05-02 08:53:11 -06:00
Connor McArthur
ade108f01c
changelog
2019-05-02 10:32:03 -04:00
Jacob Beck
6b08fd5e8d
Merge pull request #1432 from convoyinc/dev/0.13.1
...
Add MaterializedView relation type, update Snowflake adapter
2019-05-02 07:27:11 -06:00
Jacob Beck
3c8bbddb5f
Merge pull request #1413 from fishtown-analytics/feature/stub-adapter-in-parsing
...
Stub out methods at parse time
2019-05-01 10:39:49 -06:00
Jacob Beck
4c02b4a6c3
Make an stderr handler available as well and provide a way to swap between them
2019-05-01 07:11:29 -06:00
Adrian Kreuziger
786726e626
Add MaterializedView relation type, update Snowflake adapter
2019-04-30 09:54:37 -07:00
Drew Banin
1f97fe463e
Merge branch 'dev/wilt-chamberlain' into snowflake_create_or_replace
2019-04-30 12:27:25 -04:00
Jacob Beck
5a3e3ba90f
Merge pull request #1402 from fishtown-analytics/fix/quote-databases-properly
...
Quote databases properly (#1396 )
2019-04-30 10:15:37 -06:00
Jacob Beck
154aae5093
Merge pull request #1410 from fishtown-analytics/feature/test-severity
...
Add a "severity" for tests (#1005 )
2019-04-30 10:15:15 -06:00
Jacob Beck
3af88b0699
Update PackageListing to handle git packages having or not having .git
...
First write wins, on the assumption that all matching URLs are valid
2019-04-30 10:12:50 -06:00
Jacob Beck
0fb620c697
Merge pull request #1416 from fishtown-analytics/feature/strict-undefined
...
make undefined strict when not capturing macros (#1389 )
2019-04-30 09:39:29 -06:00
Jacob Beck
7d66965d0b
Merge branch 'dev/wilt-chamberlain' into feature/stub-adapter-in-parsing
2019-04-30 08:40:29 -06:00
Jacob Beck
acca6a7161
Merge pull request #1420 from fishtown-analytics/fix/postgres-text-types
...
Fix postgres text handling (#781 )
2019-04-30 08:35:07 -06:00
Jacob Beck
ad2f228048
Merge pull request #1429 from fishtown-analytics/fix/vars-in-disabled-models
...
Fix: missing vars in disabled models fail compilation (#434 )
2019-04-30 08:34:28 -06:00
Drew Banin
3a7dcd9736
Merge branch 'dev/wilt-chamberlain' into snowflake_create_or_replace
2019-04-30 09:44:43 -04:00
Drew Banin
ca15b44d0f
Update core/dbt/clients/jinja.py
...
Co-Authored-By: beckjake <beckjake@users.noreply.github.com >
2019-04-30 07:35:32 -06:00
Jacob Beck
bf9c466855
Handle warnings vs errors
...
raise errors in strict mode
log warnings on warning states
Add tests
2019-04-29 19:58:05 -06:00
Jacob Beck
abcbacaf69
Implement test failure severity levels
...
A small refactor to make test parsing easier to modify
add concept of test modifier kwargs, pass them through to config
plug the severity setting into test result handling
Update existing tests
Add integration tests
severity settings for data tests, too
2019-04-29 19:57:07 -06:00
Jacob Beck
ffceff7498
Merge branch 'dev/wilt-chamberlain' into feature/stub-adapter-in-parsing
2019-04-29 19:51:37 -06:00
Jacob Beck
25ac1db646
make undefined strict when not capturing
...
Fix a number of tests and some table/view materialization issues
2019-04-29 19:46:14 -06:00
Jacob Beck
c6d6dae352
PR feedback: Make a RedshiftColumn, make the RedshiftAdapter use it
2019-04-29 19:33:26 -06:00
Jacob Beck
aa4f771df2
Merge pull request #1419 from fishtown-analytics/feature/destroy-non-destructive
...
remove non-destructive mode (#1415 )
2019-04-29 19:28:00 -06:00
Jacob Beck
4715ad9009
Merge pull request #1426 from fishtown-analytics/feature/allow-null-vars
...
Allow vars to be set to null and differentiate them from unset vars (#608 )
2019-04-29 19:22:24 -06:00
Jacob Beck
a4e5a5ac78
Merge pull request #1425 from fishtown-analytics/fix/propagate-undefined-to-calls
...
Propagate ParserMacroCapture undefineds into the calls (#1424 )
2019-04-29 18:53:27 -06:00
Jacob Beck
f587efde60
Merge pull request #1427 from fishtown-analytics/feature/better-schema-test-errors
...
Improve invalid test warnings/errors (#1325 )
2019-04-29 18:33:54 -06:00
Jacob Beck
d57f4c54d8
at parse time, return None for any missing vars
2019-04-29 14:39:08 -06:00
Jacob Beck
b9c74e0b07
Improve invalid test warnings/errors
2019-04-29 14:08:39 -06:00
Jacob Beck
aebefe09b5
Allow vars to be set to null and differentiate them from unset vars
2019-04-29 13:32:42 -06:00
Jacob Beck
78c13d252e
Propagate ParserMacroCapture undefineds into the calls
...
Fix an issue when users attempt to use the results of missing macro calls
Add a test
2019-04-29 08:33:10 -06:00
Jacob Beck
8270c85ffd
Merge pull request #1421 from fishtown-analytics/fix/dbt-debug-connect
...
fix dbt debug connections (#1422 )
2019-04-29 07:10:13 -06:00
Bastien Boutonnet
7a2279e433
move unique key workaround to snowflake macro
2019-04-28 09:59:17 -07:00
Bastien Boutonnet
3ef519d139
todo and comments clean up
2019-04-28 07:43:16 -07:00
Bastien Boutonnet
85eac05a38
cleaner select
2019-04-28 07:33:49 -07:00
Bastien Boutonnet
8af79841f7
remove non-destructive logic
2019-04-28 07:31:14 -07:00
Bastien Boutonnet
afe236d9ac
cleaning up some commented out stuff
2019-04-27 10:25:26 -07:00
Bastien Boutonnet
90f8e0b70e
Revert "Revert "Merge branch 'dev/wilt-chamberlain' into snowflake_create_or_replace""
...
This reverts commit 4f62978de5 .
2019-04-27 10:18:30 -07:00
Bastien Boutonnet
0432c1d7e3
conflic resolve
2019-04-27 10:11:59 -07:00
Bastien Boutonnet
08820a2061
fixing my jetlagged introduced bugs
2019-04-27 08:15:23 -07:00
Bastien Boutonnet
4f62978de5
Revert "Merge branch 'dev/wilt-chamberlain' into snowflake_create_or_replace"
...
This reverts commit 3ab8238cfb , reversing
changes made to 43a9db55b1 .
2019-04-27 06:46:41 -07:00
Bastien Boutonnet
3ab8238cfb
Merge branch 'dev/wilt-chamberlain' into snowflake_create_or_replace
2019-04-27 15:09:36 +02:00
Bastien Boutonnet
43a9db55b1
quick todo marker
2019-04-27 14:35:10 +02:00
Jacob Beck
08fdcad282
prevent getting this helper method collected as a test
2019-04-26 13:50:13 -06:00
Jacob Beck
7df6e0dc68
Fix postgres text handling
...
Fix incorrect maximum text field length on redshift
On postgres, pass through text without size information to fix archival of long text fields
Add a test that makes sure postgres archives work with very long text entries
Fix tests
Add a redshift test
2019-04-26 13:50:03 -06:00
Bastien Boutonnet
5c1c5880b6
more explicit comments and quick formatting
2019-04-26 20:10:12 +02:00
Bastien Boutonnet
f99efbf72e
remove references to temp and backup relations
2019-04-26 20:04:21 +02:00
Jacob Beck
e90b60eecd
fix dbt debug connections
2019-04-26 11:14:24 -06:00
Jacob Beck
1205e15be2
remove non-destructive mode
2019-04-26 10:14:27 -06:00
Jacob Beck
32f39f35f6
Merge pull request #1417 from fishtown-analytics/feature/use-pytest
...
nosetest -> pytest
2019-04-26 08:15:21 -06:00
Bastien Boutonnet
9591b86430
(PR fdbk) rm extra macro
2019-04-26 12:43:11 +02:00
Jacob Beck
b54c6023eb
on windows, run via "python -m pytest"
2019-04-25 22:13:12 -06:00
Jacob Beck
00ba5d36b9
nosetest -> pytest
2019-04-25 21:51:43 -06:00
Jacob Beck
89eeaf1390
Merge branch 'dev/wilt-chamberlain' into feature/stub-adapter-in-parsing
2019-04-25 21:48:02 -06:00
Jacob Beck
3f18b93980
Merge pull request #1398 from fishtown-analytics/feature/archive-no-create-existing-schema
...
archives: do not create existing schema (#758 )
2019-04-25 21:42:58 -06:00
Jacob Beck
96cb056ec9
Merge pull request #1361 from fishtown-analytics/feature/archive-blocks-as-regex
...
Feature/archive blocks
2019-04-25 21:42:20 -06:00
Jacob Beck
1042f1ac8b
fix some merged-in flake8 failures
2019-04-25 14:58:29 -06:00
Jacob Beck
dd232594a5
Merge branch 'dev/wilt-chamberlain' into feature/archive-blocks-as-regex
2019-04-25 14:48:49 -06:00
Jacob Beck
5762e5fdfb
Merge pull request #1406 from fishtown-analytics/fix/run-operation-return-codes
...
Fix run operation return codes (#1377 )
2019-04-25 08:19:11 -06:00
Jacob Beck
0f1c154a1a
create a decorator for stubbing out methods at parse time
...
Includes some unit tests
Update integration tests to handle the fact that sometimes we now fail at runtime
2019-04-24 14:36:16 -06:00
Jacob Beck
ad1fcbe8b2
Merge pull request #1412 from fishtown-analytics/fix/insensitive-list-schemas
...
use ilike instead of = for database when listing schemas (#1411 )
2019-04-24 12:02:04 -06:00
Jacob Beck
877440b1e6
use ilike instead of = for database when listing schemas
2019-04-24 10:50:14 -06:00
Jacob Beck
ca02a58519
PR feedback: Add a clear_transaction call
2019-04-23 20:25:18 -06:00
Jacob Beck
2834f2d8b6
update test.env.sample
2019-04-23 19:39:44 -06:00
Jacob Beck
cc4f285765
if the schema exists, do not try to create it
2019-04-23 14:59:50 -06:00
Jacob Beck
2efae5a9c3
give bigquery list_schemas/check_schema_exist macros
2019-04-23 14:59:50 -06:00
Jacob Beck
416cc72498
Implmement check_cols
...
Contracts: some anyOf shenanigans to add support for check_cols
Macros: split apart archive selection, probably too much copy+paste
Legacy: Archive configs now include a "timestamp" strategy when parsed from dbt_project.yml
Add integration tests
fix aliases test
Unquote columns in archives
handle null columns
attr -> use_profile
2019-04-23 14:34:24 -06:00
Jacob Beck
d66584f35c
Update jsonschema and go from Draft 4 to Draft 7
2019-04-23 14:34:24 -06:00
Jacob Beck
2b80d7ad8d
move column-related things into adapters where they belong
2019-04-23 14:34:24 -06:00
Jacob Beck
be3445b78a
get archive blocks working
...
tests
fix event tracking test
Fix print statements
make archives not inherit configs from models
archive now uses the name/alias properly for everything instead of target_table
skip non-archive blocks in archive parsing instead of raising
make archives ref-able
- test for archive ref, test for archive selects
raise a more useful message on incorrect archive targets
add "--models" and "--exclude" arguments to archives
- pass them through to selection
- change get_fqn to take a full node object, have archives use that so selection behaves well
- added tests
Improve error handling on invalid archive configs
Added a special archive-only node that has extra config restrictions
add tests for invalid archive config
2019-04-23 14:33:44 -06:00
Jacob Beck
ab63042dfa
archive-paths support, wire up the block parser
...
raise on non-archive during parsing
break archive materialization
2019-04-23 14:33:12 -06:00
Jacob Beck
af8622e8ff
flake8, pep8, unit tests
2019-04-23 14:32:23 -06:00
Jacob Beck
53d083ec58
fix this for real in a way that will make me not break it again
2019-04-23 14:30:59 -06:00
Jacob Beck
32f74b60ef
Merge pull request #1408 from fishtown-analytics/fix/remove-sql_where-from-subquery
...
Remove sql_where
2019-04-23 07:59:35 -06:00
Jacob Beck
0885be1dc0
Remove sql_where, removing an unnecessary subquery in the process
2019-04-22 11:39:54 -06:00
Jacob Beck
8b58b208ca
add quote policy to Relation.create calls
2019-04-22 11:29:07 -06:00
Jacob Beck
3188aeaac4
More tests
...
Add an already_exists call to a test that requires an alternative database
Make the alternative database on snowlfake be one that must be quoted
2019-04-22 11:29:07 -06:00
Bastien Boutonnet
e83edd30de
fixme/todo regarding non destructive flag
2019-04-22 16:02:03 +02:00
Bastien Boutonnet
04333699a0
remove testing logging messages
2019-04-22 15:49:59 +02:00
Bastien Boutonnet
95c9f76e32
remove snowflake__ direct call in incremental
2019-04-22 11:10:02 +02:00
Bastien Boutonnet
2830b6a899
make default create or replace macro to allow snowflake adapter pick it up
2019-04-22 11:06:58 +02:00
Bastien Boutonnet
54c02ef4b4
some logging and temp call workaround
2019-04-22 01:32:07 +02:00
Bastien Boutonnet
dacce7c864
fix insert cols call and temp workaround call of snowflake
2019-04-22 01:27:25 +02:00
Jacob Beck
08c5f9aed8
no automatic transactions
2019-04-21 16:43:52 -06:00
Bastien Boutonnet
fb26ce5c24
Merge branch 'snowflake_create_or_replace' of github.com:bastienboutonnet/dbt into snowflake_create_or_replace
2019-04-20 19:32:40 +01:00
Bastien Boutonnet
91d869e61a
revert test
2019-04-20 19:29:01 +01:00
Bastien Boutonnet
d168bdd0c8
Merge branch 'snowflake_create_or_replace' of github.com:bastienboutonnet/dbt into snowflake_create_or_replace
2019-04-20 19:25:41 +01:00
Bastien Boutonnet
6a104c1938
test
2019-04-20 19:25:16 +01:00
Bastien Boutonnet
2d5525e887
add some logging
2019-04-20 19:09:46 +01:00
Bastien Boutonnet
a35ad186e3
implement insert when no unique key and full refresh solution
2019-04-20 18:13:37 +01:00
Jacob Beck
dd469adf29
Merge pull request #1407 from fishtown-analytics/feature/wrap-models-in-jinja-tags
...
add a flag to wrap models/tests in jinja blocks
2019-04-18 11:48:22 -06:00
Jacob Beck
4ffc5cbe6a
PR feedback
2019-04-18 11:17:26 -06:00
Jacob Beck
f3449dcfcb
add a flag to wrap models/tests in jinja blocks, to see what happens
2019-04-18 11:17:26 -06:00
Jacob Beck
4e8c7b9216
add a test and a comment about an odd edge case in do-block parsing
2019-04-18 09:27:52 -06:00
Jacob Beck
473078986c
PR feedback: run_unsafe -> _run_unsafe
2019-04-18 09:05:23 -06:00
Jacob Beck
5b74c58a43
add tests
2019-04-17 11:26:33 -06:00
Jacob Beck
a72a4e1fcb
Acquire a connection before executing the macro, and commit after
2019-04-17 11:26:33 -06:00
Jacob Beck
13dd72029f
run-operation fixes
...
Make dbt run-operation actually function at all with the RPC changes
On exceptions that occur outside of actual macro execution, catch them and return failure appropriately
2019-04-17 11:26:32 -06:00
Jacob Beck
fc4fc5762b
add a pretty janky data test to expose quoting issues
2019-04-16 13:51:34 -06:00
Jacob Beck
d515903172
make PostgresAdapter.verify_database handle names that resolve to the same thing
2019-04-16 13:51:33 -06:00
Jacob Beck
97a6a51bec
Quote databases when we list them
...
Fix a copy+paste error that broke database quoting configuration
2019-04-16 13:51:33 -06:00
Bastien Boutonnet
9222c79043
implement create or replace in table mater
2019-04-16 17:53:52 +02:00
Bastien Boutonnet
38254a8695
make create or replace snowflake macro
2019-04-16 17:53:31 +02:00
Jacob Beck
9b1aede911
Fix a number of bugs
...
After we find the start of a comment block, advance immediately
- this is so we do not mistake "{#}" as both open and close of comment
support do/set statements (no {% enddo %}/{% endset %})
fix some edge-case bugs around quoting
fix a bug around materialization parsing
2019-04-16 09:52:27 -06:00
Jacob Beck
ac71888236
add tests
...
test for comment blocks where the first character of the comment is "}"
do/set tests
an even trickier test case
2019-04-16 09:45:28 -06:00
Jacob Beck
3f31b52daf
Merge pull request #1388 from fishtown-analytics/fix/bigquery-missing-model-names
...
pass the model name along in get_relations
2019-04-11 07:28:22 -06:00
Connor McArthur
e3230aad55
Merge pull request #1391 from fishtown-analytics/add-logging-to-dbt-clean
...
Add logging to dbt clean
2019-04-08 09:31:20 -05:00
emilielimaburke
ac40aa9b02
fix merge conflict
2019-04-08 09:27:08 -05:00
Emilie Lima Schario
fa480e61a1
update docs with steps
2019-04-08 09:25:14 -05:00
Jacob Beck
c19644882b
add unit tests
2019-04-05 10:22:54 -06:00
Jacob Beck
e29eccd741
pass the model name along in get_relations
2019-04-04 17:50:42 -06:00
Jacob Beck
4dd80567e1
Merge pull request #1380 from fishtown-analytics/feature/rpc-ps
...
RPC server task management
2019-04-03 11:46:37 -06:00
Jacob Beck
2654c79548
PR feedback
2019-04-03 10:01:36 -06:00
Jacob Beck
3b357340fd
skip the timing assert on python 2.x
2019-04-02 15:15:43 -06:00
Jacob Beck
6c8e74bac9
tests, fight with (test-only?) deadlocks
...
when adding more threads stops helping, add more sleeps
2019-04-02 15:15:37 -06:00
Jacob Beck
182714b6b8
handle ctrl+c during parsing, etc
2019-04-02 15:14:28 -06:00
Jacob Beck
8410be848f
fix the methods list
2019-04-02 15:14:28 -06:00
Jacob Beck
3f9b9962c3
add "ps" and "kill" commands, and track tasks in flight
...
proper cancel support
Refactor rpc server logic a bit
fix an issue in query cancelation where we would cancel ourselves
fix exception handling misbehavior
2019-04-02 15:14:28 -06:00
Jacob Beck
ec1f4bc33d
fix bad add_query calls
...
also fix unit tests
2019-04-02 15:14:28 -06:00
Jacob Beck
f2a0d36b34
when a dbt RuntimeException is raised inside the exception handler, re-raise it instead of wrapping it
2019-04-02 15:14:28 -06:00
Jacob Beck
fc2b86df4f
rearrange some path handling in the rpc server task
2019-04-02 15:14:28 -06:00
Jacob Beck
0cd0792b65
Merge pull request #1378 from fishtown-analytics/ci/add-build-tag
...
build on all dev branches, build on any branch named starting with "pr/"
2019-04-02 14:27:27 -06:00
Jacob Beck
122ee5ab7d
Merge pull request #1381 from fishtown-analytics/fix/report-compiled-node-on-error
...
Fix/report compiled node on error
2019-04-02 14:24:32 -06:00
Darren Haken
8270ef71b2
Merge branch 'dev/wilt-chamberlain' of github.com:fishtown-analytics/dbt into dev/stephen-girard
2019-04-01 14:31:33 +01:00
Darren Haken
d59a13079f
Merge remote-tracking branch 'upstream/dev/stephen-girard' into dev/stephen-girard
2019-04-01 14:30:59 +01:00
Jacob Beck
ed59bd22f3
actually override macros...
...
fix macro overrides
fix tests that masked the broken macro overrides
2019-03-29 09:03:54 -06:00
Jacob Beck
8d32c870fc
attach nodes to exceptions and pass compiled sql into the error messages
2019-03-29 08:19:30 -06:00
Jacob Beck
bea2d4fb34
Merge pull request #1373 from fishtown-analytics/fix/rpc-ephemeral-clean
...
Support ephemeral nodes (#1368 )
2019-03-27 13:02:29 -06:00
Jacob Beck
fcb97bf78a
Merge pull request #1363 from roverdotcom/feature/alias-macro
...
Adding a generate_alias_name macro
2019-03-27 08:19:34 -06:00
Jacob Beck
161a78dc23
build on all dev branches or branches that start with "pr/"
2019-03-26 15:52:12 -06:00
Brandyn Lee
4b7bddb481
generate_macro update
2019-03-26 13:32:56 -07:00
Brandyn Lee
0879b1b38b
A more verbose function
2019-03-26 13:29:08 -07:00
Brandyn Lee
8bf81a581a
Fix integration tests
2019-03-26 13:29:08 -07:00
Brandyn Lee
759da58648
Fix unit test, node.name
2019-03-26 13:29:08 -07:00
Brandyn Lee
ec4a4fe7df
Add second integration test and pass parsed_node through macro
2019-03-26 13:29:08 -07:00
Brandyn Lee
4dedd62aea
Minimal testing
2019-03-26 13:29:08 -07:00
Brandyn Lee
cf4030ed94
Get unit testing working
2019-03-26 13:29:08 -07:00
Brandyn Lee
35df887307
Get get_customer_alias working.
2019-03-26 13:29:08 -07:00
Brandyn Lee
d41adaa51b
Copying all the customer schema code
2019-03-26 13:29:08 -07:00
Jacob Beck
9373a45870
add integration test
2019-03-26 07:28:37 -06:00
Jacob Beck
a2db88c9c3
attach nodes to results during processing, but not for serialization
2019-03-26 07:25:35 -06:00
Jacob Beck
a26d7bf9e8
pass along the compiled manifest instead of pitching it away
2019-03-26 07:25:34 -06:00
Jacob Beck
4225047b06
Merge pull request #1375 from fishtown-analytics/feature/inline-macros-models
...
Support macros in execute/compile tasks (#1372 )
2019-03-26 07:16:51 -06:00
Jacob Beck
0a9ed9977b
include top-level raw/comment blocks
2019-03-25 17:30:49 -06:00
Jacob Beck
fc1b4ce88e
add all non-macro toplevel blocks to the sql so things like "if" work
2019-03-25 15:30:21 -06:00
Jacob Beck
6295c96852
get macros from the body of the sql data instead of a separate parameter
2019-03-25 15:30:21 -06:00
Jacob Beck
73418b5c16
make the block lexer include raw toplevel data
2019-03-25 15:30:21 -06:00
Jacob Beck
98d530f0b1
Jinja block parsing/lexing implemented
2019-03-25 14:20:24 -06:00
Drew Banin
e45ed0ed8c
Merge pull request #1366 from emilieschario/fix_profile_link_issue_1344
...
Update broken profiles.yml link
2019-03-24 12:07:58 -04:00
emilielimaburke
fc04e2db89
updates link
2019-03-24 11:59:20 -04:00
Drew Banin
cfaacc5a76
Merge pull request #1364 from fishtown-analytics/fix/redshift-test-env-password
...
fix sample test env password
2019-03-23 14:43:24 -04:00
Drew Banin
b91c3edb16
fix sample test env password
2019-03-21 19:29:03 -04:00
Connor McArthur
0a503a0bed
try to resolve merge
2019-03-21 15:13:11 -04:00
Connor McArthur
015e4d66b2
resolve connection naming conflict
2019-03-21 15:12:00 -04:00
Connor McArthur
9e36ebdaab
Merge branch '0.13.latest' of github.com:fishtown-analytics/dbt
2019-03-21 13:27:24 -04:00
Connor McArthur
da4c135e23
Merge pull request #1359 from fishtown-analytics/dev/stephen-girard
...
0.13.0 (Stephen Girard)
2019-03-21 13:26:58 -04:00
Drew Banin
588851ac1c
Update CHANGELOG.md
2019-03-21 11:16:29 -04:00
Drew Banin
2b7d7061f9
Update CHANGELOG.md
2019-03-21 11:15:03 -04:00
Connor McArthur
24bc3b6d76
Bump version: 0.13.0rc1 → 0.13.0
2019-03-21 11:05:28 -04:00
Connor McArthur
cd52a152f6
update RELEASE instructions, add script to build all the source distributions
2019-03-21 11:04:51 -04:00
Connor McArthur
3ecf8be873
add RC bugfixes
2019-03-21 10:50:03 -04:00
Connor McArthur
f38466db11
Merge pull request #1360 from fishtown-analytics/0.13.0-changelog
...
0.13.0 changelog
2019-03-21 10:40:34 -04:00
Drew Banin
3b8d5c0609
Merge pull request #1356 from fishtown-analytics/fix/concurrent-transaction-fresness
...
fix for error-ed out transactions on redshift
2019-03-20 16:49:14 -04:00
Drew Banin
60539aaa56
fix for errored out transactions on redshift
2019-03-20 14:55:54 -04:00
Drew Banin
6d53e67670
Update parsed.py
...
pep8
2019-03-19 15:37:13 -04:00
Drew Banin
9771e63247
Merge branch 'dev/stephen-girard' into dev/stephen-girard
2019-03-19 15:15:02 -04:00
Jacob Beck
02c9bcabe0
Merge pull request #1355 from fishtown-analytics/fix/out-of-order-execution-on-model-select
...
Fix out of order execution on model select (#1354 )
2019-03-19 07:34:52 -06:00
Jacob Beck
69c8a09d43
Use the transitive closure to calculate the graph
...
Don't maintain the links manually while removing nodes
Just take the transitive closure and remove all nodes
2019-03-18 16:49:46 -06:00
Jacob Beck
9ae229a0d5
ugh, forgot to remove this I guess
2019-03-18 15:51:26 -06:00
Jacob Beck
38921fad17
tests
2019-03-18 15:48:22 -06:00
Jacob Beck
633858a218
When building the graph underlying the graph queue, preserve transitive dependencies while removing the skipped nodes
2019-03-18 15:08:51 -06:00
Drew Banin
70262b38f8
Merge pull request #1353 from fishtown-analytics/fix/issue-1352
...
possibly reckless fix for #1352
2019-03-18 09:30:46 -04:00
Jacob Beck
f96dedf3a9
redshift can just change this on you apparently
2019-03-16 12:56:10 -04:00
Drew Banin
1ce0493488
possibly reckless fix for #1352
2019-03-15 15:08:48 -04:00
Jacob Beck
027a0d2ee6
Merge pull request #1341 from fishtown-analytics/feature/rpc-improve-dbt-exceptions
...
RPC: Error handling improvements
2019-03-12 16:17:50 -06:00
Jacob Beck
9c8e08811b
redshift can just change this on you apparently
2019-03-12 15:09:07 -06:00
Jacob Beck
c1c09f3342
Merge pull request #1348 from fishtown-analytics/feature/rpc-with-macros
...
RPC: macros
2019-03-12 14:00:30 -06:00
Drew Banin
05b82a22bc
Merge pull request #1349 from fishtown-analytics/fix/new-logo
...
Use new logo
2019-03-11 21:33:45 -04:00
Drew Banin
067aa2ced0
replace logo
2019-03-11 21:32:40 -04:00
Drew Banin
f18733fd09
Add files via upload
2019-03-11 21:31:41 -04:00
Drew Banin
a981f657ec
Merge pull request #1347 from fishtown-analytics/fix/warn-error-flag
...
re-add --warn-error flag
2019-03-11 20:48:15 -04:00
Jacob Beck
81426ae800
add optional "macros" parameter to dbt rpc calls
2019-03-11 18:25:17 -06:00
Drew Banin
4771452590
Merge pull request #1346 from fishtown-analytics/fix/on-run-hooks-in-tests
...
[Stephen Girard] fix for on-run- hooks running in tests
2019-03-11 20:07:46 -04:00
Drew Banin
10bfaf0e4b
Update CHANGELOG.md
2019-03-11 19:56:57 -04:00
Drew Banin
9e25ec2f07
Update CHANGELOG.md
2019-03-11 19:55:57 -04:00
Drew Banin
90fb908376
re-add --warn-error flag
2019-03-11 19:42:14 -04:00
Drew Banin
a08c0753e7
fix for on-run- hooks running in tests
2019-03-11 13:01:22 -04:00
Jacob Beck
fc22cb2bf0
when encoding json, handle dates and times like datetimes
2019-03-08 13:28:44 -07:00
Jacob Beck
fbaae2e493
fix Python 2.7
2019-03-08 12:02:51 -07:00
Jacob Beck
c86390e139
use notice logging for "Found x models, ...", change a couple other levels
2019-03-08 10:54:50 -07:00
Jacob Beck
d890642c28
add NOTICE level logging, make log messages richer types
2019-03-08 10:54:10 -07:00
Jacob Beck
6620a3cd90
wrap all context-raised exceptions in node info
...
Fixes "called by <Unknown>"
2019-03-08 10:15:16 -07:00
Jacob Beck
7e181280b3
PR feedback: QueueMessageType class, remove extra assignments
2019-03-08 08:19:50 -07:00
Jacob Beck
53499e6b14
Merge branch 'dev/stephen-girard' into dev/wilt-chamberlain
2019-03-08 08:06:18 -07:00
Jacob Beck
3f948ae501
Error handling improvements
...
All dbt errors now have proper error codes/messages
The raised message at runtime ends up in result.error.data.message
The raised message type at runtime ends up in result.error.data.typename
result.error.message is a plaintext name for result.error.code
dbt.exceptions.Exception.data() becomes result.error.data
Collect dbt logs and make them available to requests/responses
2019-03-08 07:25:21 -07:00
Jacob Beck
2090887a07
Merge pull request #1336 from fishtown-analytics/feature/one-connection-per-thread
...
per-thread connections
2019-03-08 06:21:53 -07:00
Jacob Beck
a7bfae061c
Merge pull request #1342 from fishtown-analytics/fix/temp-relations-from-modelname
...
Prefix temp relations with model name instead of alias (#1321 )
2019-03-08 06:17:52 -07:00
Jacob Beck
fb1926a571
use model name instead of alias for temp tables to ensure uniqueness
2019-03-07 17:08:40 -07:00
Jacob Beck
c215158d67
PR feedback, clean up associated connection holding
2019-03-06 19:47:12 -07:00
Connor McArthur
74152562fe
Bump version: 0.13.0a2 → 0.13.0rc1
2019-03-06 17:44:57 -05:00
Jacob Beck
e2af871a5a
per-thread connections
...
parsing now always opens a connection, instead of waiting to need it
remove model_name/available_raw/etc
2019-03-06 12:10:22 -07:00
Jacob Beck
2ad116649a
Merge branch 'dev/stephen-girard' into dev/wilt-chamberlain
2019-03-06 07:18:06 -07:00
Jacob Beck
03aa086e0b
Merge pull request #1338 from fishtown-analytics/fix/jeb-snowflake-source-quoting
...
Fix snowflake source quoting / information_schema uses
2019-03-06 06:59:13 -07:00
Jacob Beck
a335857695
PR feedback
2019-03-05 21:47:37 -07:00
Jacob Beck
95a88b9d5d
PR feedback
2019-03-05 18:12:19 -07:00
Jacob Beck
2501783d62
fix macro kwargs
2019-03-05 18:02:13 -07:00
Jacob Beck
7367f0ffbd
Merge remote-tracking branch 'origin/snowflake-unquote-db-by-default-2' into fix/jeb-snowflake-source-quoting
2019-03-05 17:17:46 -07:00
Jacob Beck
088442e9c1
test fixes
2019-03-05 17:12:02 -07:00
Jacob Beck
ec14c6b2dc
initial work, unit tests
2019-03-05 17:11:29 -07:00
Drew Banin
7eb033e71a
fix incorrect schema filtering logic
2019-03-05 17:54:29 -05:00
Drew Banin
1a700c1212
Merge pull request #1330 from fishtown-analytics/fix/handle-unexpected-snapshot-values
...
[Stephen Girard] handle unexpected max_loaded_at types
2019-03-05 17:22:16 -05:00
Drew Banin
78fbde0e1f
Merge branch 'dev/stephen-girard' of github.com:fishtown-analytics/dbt into fix/handle-unexpected-snapshot-values
2019-03-05 14:43:50 -05:00
Darren Haken
a30cc5e41e
Add persist_docs to dbt's contract
2019-03-05 15:15:30 +00:00
Darren Haken
804a495d82
Fix BQ integration tests by adding persist_docs: {}
2019-03-05 14:51:00 +00:00
Jacob Beck
0a4eea4388
Merge pull request #1301 from fishtown-analytics/feature/rpc-tasks
...
RPC server (#1274 )
2019-03-05 06:35:03 -07:00
Jacob Beck
8471ce8d46
Merge branch 'dev/wilt-chamberlain' into feature/rpc-tasks
2019-03-04 21:07:56 -07:00
Jacob Beck
f9b1cf6c1c
Merge pull request #1314 from fishtown-analytics/azure-pipelines-2
...
Set up CI with Azure Pipelines
2019-03-04 21:06:04 -07:00
Drew Banin
22a2887df2
add missing import
2019-03-04 22:50:23 -05:00
Jacob Beck
02e88a31df
Merge branch 'dev/wilt-chamberlain' into feature/rpc-tasks
2019-03-04 19:55:51 -07:00
Jacob Beck
98d5bc1285
PR feedback
2019-03-04 19:50:24 -07:00
Jacob Beck
436815f313
Merge branch 'dev/stephen-girard' into azure-pipelines-2
2019-03-04 19:50:16 -07:00
Drew Banin
328ce82bae
fix unit tests
2019-03-04 20:43:37 -05:00
Drew Banin
d39a048e6e
pr feedback
2019-03-04 20:39:31 -05:00
Drew Banin
67b56488d3
0.13.0 fixes around database quoting and rendering
...
- do not quote snowflake database identifiers by default
- do not find relations in source schemas in list_relations
- do not render database names in stdout if a custom database is not specified
2019-03-04 09:09:29 -05:00
Drew Banin
07397edd47
handle unexpected loaded_at field types
2019-03-03 17:06:34 -05:00
Darren Haken
1e3bdc9c06
Fix unit tests by adding persist_docs: {}
2019-03-02 17:36:28 +00:00
Darren Haken
12bfeaa0d3
Merge remote-tracking branch 'upstream/dev/stephen-girard' into dev/stephen-girard
2019-03-02 17:19:36 +00:00
Blake Blackwell
56801f9095
Adding changes based on Drew's recommendations
2019-03-02 05:28:09 -06:00
Drew Banin
54b0b38900
Merge pull request #1328 from fishtown-analytics/feature/operations-actually
...
Feature/operations actually
2019-03-01 11:27:42 -05:00
Drew Banin
e4660969cf
Merge pull request #1324 from fishtown-analytics/fix/snowflake-rename
...
Fix for alter table ... rename statements on Snowflake
2019-02-28 14:59:57 -05:00
Drew Banin
1090a1612a
add run-operation subtask
2019-02-28 11:16:14 -05:00
Drew Banin
f4baba8cc1
Merge pull request #1327 from fishtown-analytics/fix/relocate-event-json-schemas
...
relocate events dir back to root of repo
2019-02-28 10:12:44 -05:00
Darren Haken
28fa237f87
Add tests for the BQ description in the relations
2019-02-28 14:18:44 +00:00
Drew Banin
f19f0e8193
( fixes #1319 ) relocate events dir back to root of repo
2019-02-27 22:46:01 -05:00
Drew Banin
72d6ee2446
fix tests
2019-02-27 13:24:32 -05:00
Drew Banin
f8dfe48653
( fixes #1313 ) Show sources in resource count list during compilation
2019-02-26 13:18:30 -05:00
Drew Banin
9c9c0d991a
( fixes #1316 ) Fix for alter table rename on Snowflake tables, added tests
2019-02-26 13:08:28 -05:00
Jacob Beck
8d2cb5fdf1
more newlines
2019-02-22 15:03:22 -07:00
Jacob Beck
1486796973
on windows, host on "localhost" instead of "database"
2019-02-22 14:07:59 -07:00
Jacob Beck
1300f8f49f
Set up CI with Azure Pipelines
...
Add yml file
chunk up windows tests in tox.ini
add missing mock.patch calls
fix path issues in docs tests
better error messaging on mystery exceptions
ENOEXEC handling on Windows
Handle pipelines insisting on cloning with crlf line endings
set up postgres service for the postgres tests
remove appveyor
2019-02-22 13:44:37 -07:00
Jacob Beck
29e9c63e94
the RPC tests fail on windows, just skip them
2019-02-21 16:59:05 -07:00
Jacob Beck
4bda6769c5
fix an error handling bug I introduced
2019-02-21 15:29:10 -07:00
Jacob Beck
dc5c59b40b
PR feedback
...
argument parsing fixes
change the table to a list of columns + list of rows
2019-02-20 15:06:05 -07:00
Jacob Beck
a90ef2504e
PR feedback
...
Fix python 2.7
remove TODO
remove useless file I added by accident
close Pipe members
Give RPC its own logger, include remote addrs
2019-02-20 07:45:52 -07:00
Jacob Beck
0f3967e87d
add tests, put them in sources to re-use all the source work
2019-02-19 14:41:48 -07:00
Jacob Beck
1a0df174c9
Implement the RPC server
...
- make tasks all have a "from_args" that handles initializing correct config type, etc
- make it possible to process a single node's refs at a time
- Make remote run/compile tasks + rpc server task, wire them up
- add ref() and source() support, and vestigial doc() support
- refactor results a bit to support new result behavior
- don't write to the filesystem on requests
- handle uniqueness issues
2019-02-19 10:43:27 -07:00
Darren Haken
a1b5375e50
Add AttributionError to except block
2019-02-19 15:59:12 +00:00
Darren Haken
101fd139c7
Add try catch for the updating config value.
...
This resolves the issue that `{{ config(persist_docs=true) }}`
would not raise a useful exception.
2019-02-19 15:41:51 +00:00
Darren Haken
25d5fb1655
Add persist_docs to project level settings.
...
Change `table_options` to have better error handling.
2019-02-19 12:18:09 +00:00
Darren Haken
e672042306
Merge remote-tracking branch 'upstream/dev/stephen-girard' into dev/stephen-girard
2019-02-19 11:46:04 +00:00
Connor McArthur
42ec3f9f06
add base setup.py to .bumpversion.cfg, bump the version
2019-02-18 16:04:24 -05:00
Connor McArthur
a4fd148a80
Bump version: 0.13.0a1 → 0.13.0a2
2019-02-18 16:02:37 -05:00
Drew Banin
e9927fb09c
Merge pull request #1303 from fishtown-analytics/fix/0.13.0-tweaks
...
Fix/0.13.0 tweaks
2019-02-18 15:57:45 -05:00
Drew Banin
da31c9a708
pep8
2019-02-18 12:37:16 -05:00
Drew Banin
1be8cb8e91
add logging for registry requests
2019-02-18 12:34:20 -05:00
Drew Banin
a6ae79faf4
show db name in model result output
2019-02-18 12:30:19 -05:00
Drew Banin
61c345955e
Merge pull request #1302 from fishtown-analytics/fix/source-timestamps
...
fix for invalid timestamps returned by source freshness cmd
2019-02-18 12:27:01 -05:00
Drew Banin
ebce6da788
fix tests
2019-02-18 10:00:15 -05:00
Blake Blackwell
9772c1caeb
Adding incremental logic to Snowflake plugins
2019-02-17 19:28:46 -06:00
Darren Haken
5661855dcc
Merge remote-tracking branch 'upstream/dev/stephen-girard' into dev/stephen-girard
2019-02-16 15:22:02 +00:00
Darren Haken
11319171be
Add description for BQ tables and view relations.
...
Also make changes to table_options macro based on testing against a
real project.
2019-02-16 15:21:43 +00:00
Drew Banin
3134b59637
fix for invalid timestamps returned by source freshness cmd
2019-02-15 20:23:07 -05:00
Connor McArthur
dfb87dce38
fix long_description field in all the setup.pys
2019-02-14 15:46:23 -05:00
Connor McArthur
ce105d2350
Merge pull request #1299 from fishtown-analytics/rm-presto
...
remove presto adapter
2019-02-14 15:22:19 -05:00
Connor McArthur
e039397ab1
add setup.py for dbt
2019-02-14 14:34:43 -05:00
Connor McArthur
b54aadf968
rm presto adapter unit test
2019-02-14 14:30:55 -05:00
Connor McArthur
e9cf074b45
move presto docker stuff
2019-02-14 14:21:41 -05:00
Connor McArthur
c417c2011b
remove presto from requirements.txt
2019-02-14 14:19:58 -05:00
Connor McArthur
2c3c3c9a84
remove presto adapter
2019-02-14 14:18:30 -05:00
Drew Banin
f546390221
Update CHANGELOG.md
2019-02-14 10:00:16 -05:00
Drew Banin
40034e056f
Merge pull request #1298 from fishtown-analytics/feature/sources-in-docs-site
...
bump docs site index.html to use sources
2019-02-14 09:38:36 -05:00
Drew Banin
47e9896d54
bump docs site index.html to use sources
2019-02-14 09:38:03 -05:00
Jacob Beck
cda365f22a
Merge pull request #1297 from fishtown-analytics/fix/widen-boto3-restrictions
...
fix: Widen botocore/boto3 requirements (#1234 )
2019-02-14 07:18:59 -07:00
Jacob Beck
36e1252824
Widen botocore/boto3 requirements
...
Remove botocore/boto3 restrictions on the snowflake plugin
Increase minimum snowflake-connector-python to 1.6.12
- 1.6.12 is the most recent boto3/botocore requirements change
Increase the upper bounds on boto3/botocre to match the redshift plugin
- they match the upper bounds of snowflake-connector-python 1.6.12
2019-02-13 15:38:21 -07:00
Drew Banin
cf873d0fc5
Merge pull request #1293 from fishtown-analytics/fix/incremental-from-view
...
Fix for models which transition from views to incremental
2019-02-13 17:09:55 -05:00
Jacob Beck
026c50deb3
Merge pull request #1272 from fishtown-analytics/feature/source-freshness
...
Feature: source freshness (#1240 )
2019-02-13 14:40:34 -07:00
Jacob Beck
c5138eb30f
Merge pull request #1295 from fishtown-analytics/fix/dbt-seed-show
...
fix dbt seed --show (#1288 )
2019-02-13 13:03:00 -07:00
Jacob Beck
0bd59998c0
Merge branch 'dev/stephen-girard' into feature/source-freshness
2019-02-13 12:14:57 -07:00
Jacob Beck
7cd336081f
Merge branch 'dev/stephen-girard' into feature/source-freshness
2019-02-13 11:21:31 -07:00
Jacob Beck
47cc931b0f
Merge pull request #1291 from fishtown-analytics/feature/enhanced-model-selection
...
Enhanced model selection syntax (#1156 )
2019-02-13 11:08:19 -07:00
Jacob Beck
5462216bb3
fix dbt seed --show
...
Fix the invalid dict-stype lookup on the result of dbt seed --show
Add a --show to a test to make sure it doesn't crash dbt
2019-02-13 10:49:54 -07:00
Drew Banin
fe86615625
Merge pull request #1286 from fishtown-analytics/fix/error-url-v1-schema
...
fix error message url
2019-02-13 11:42:41 -05:00
Drew Banin
9a74abf4cc
Merge pull request #1252 from fishtown-analytics/feature/snowflake-transient-tables
...
Support transient tables on Snowflake
2019-02-13 11:42:18 -05:00
Drew Banin
2847f690f1
Merge branch 'dev/stephen-girard' into fix/incremental-from-view
2019-02-13 11:27:25 -05:00
Drew Banin
2c94e9e650
( fixes #1292 ) Check for relation type in is_incremental()
...
If the target relation is a non-table (probably a view) then dbt
should return False from the is_incremental() macro. The
materialization will drop this relation before running the model
code as a `create table as` statement, so the incremental filter
would likely be invalid.
2019-02-13 11:26:40 -05:00
Jacob Beck
95ab2ab443
fix Snowflake tests
2019-02-13 09:05:07 -07:00
Jacob Beck
3dcfa2c475
Merge branch 'dev/stephen-girard' into feature/enhanced-model-selection
2019-02-13 08:59:03 -07:00
Jacob Beck
7bab31543e
Merge pull request #1280 from fishtown-analytics/feature/werror
...
Add flag for raising errors on warnings (#1243 )
2019-02-13 08:55:14 -07:00
Jacob Beck
08c4c2a8b5
PR feedback
...
print source name on pass/warn/error lines
distinguish 'error' vs 'error stale'
2019-02-13 08:44:23 -07:00
Jacob Beck
9fcad69bf4
tests
2019-02-13 08:17:19 -07:00
Jacob Beck
b406a536a9
initial selector work
2019-02-13 08:17:13 -07:00
Jacob Beck
5e8ab9ce4a
Remove RunManager
...
Move some of RunManager into tasks
Move compile_node work into compilation
Move manifest work into the GraphLoader
Move the rest into the runners
Implement freshness calculations for sources
command: 'dbt source snapshot-freshness'
support for 4 adapters (no presto)
Integration tests
break up main.py's argument parsing
Pass the manifest along to freshness calculation
Results support for freshness
New freshness result contracts
Fix source result printing
Result contract cleanup
safe_run supports alternate result types
Fix tests to support changes in results
PR feedback:
- snowflake macro changed to always return utc
- no cte in collect_freshness
- remove extra optional arg
- fix the has_freshness check to examine if there is anything in freshness
- support error_after without warn_after and vice-versa
- snowflake: convert_timestamp -> convert_timezone
Update sources to be Relations
- update contracts
- add create_from_source
- add create_from_source calls
- fix tests
PR feedback
create_from_source forces quotes
default source schema/table from source/table names
snowflake quoting nonsense
also fix output: pass -> PASS
make seeding test 017 take 1m instead of 3m by using csv instead of sql
- source tweaks for the docs site
2019-02-13 10:07:37 -05:00
Darren Haken
562d47f12a
Make changes as per review from @drewbanin
2019-02-12 15:21:29 +00:00
Drew Banin
ab6d4d7de5
use frozenset for adapter specific configs, add bq configs
2019-02-11 13:04:58 -05:00
Drew Banin
4b9ad21e9e
fix error message url
2019-02-10 10:34:50 -05:00
Darren Haken
f4c233aeba
#1031 Add macro to add table description from schema.yml for BQ
2019-02-10 13:38:40 +00:00
Jacob Beck
90497b1e47
give deprecation its own integration test
2019-02-08 08:22:12 -07:00
Jacob Beck
9b9319cbd0
fix bigquery
...
Remove unnecessary is_incremental() from test
Fix strict mode type check
2019-02-08 08:00:16 -07:00
Jacob Beck
fe948d6805
Add flag for raising errors on warnings
...
fix strict mode
fix some strict mode asserts
add internal WARN_ERROR support and a warn_on_error function
make everything that says "warning" go through warn_or_error
add --warn-error flag to main.py
remove sql_where from tests
replace drop-existing with full-refresh in tests
(re-)add integration tests for malformed schema/source tests and strict mode
2019-02-08 08:00:09 -07:00
Jacob Beck
b17d70679f
Merge pull request #1277 from fishtown-analytics/fix/backwards-compatible-adapter-functions
...
0.13.0 backwards compatibility (#1273 )
2019-02-08 06:54:36 -07:00
Jacob Beck
5290451a65
force upgrade pip on appveyor to see if that helps, idk
2019-02-07 19:51:20 -07:00
Jacob Beck
faadb34aff
backwards compatibility work
...
create deprecation warning decorator for deprecated available methods
make already_exists just take schema/name again and direct users to get_relation
remove test that cannot fail (the deprecation does not exist!)
add a little deprecation warning check to test_simple_copy
2019-02-05 09:49:10 -07:00
Connor McArthur
314b4530d8
Merge pull request #1242 from fishtown-analytics/add-node-level-timing-info
...
add node level timing info to run_results.json and to tracking
2019-02-01 16:45:51 -05:00
Connor McArthur
843d342137
fix event tracking tests
2019-02-01 14:34:28 -05:00
Connor McArthur
f0981964f3
upgrade run_model iglu schema to 1-0-1
2019-02-01 13:59:53 -05:00
Jacob Beck
da409549d4
Merge pull request #1262 from fishtown-analytics/feature/source-table-selection
...
Add source selector support (#1256 )
2019-01-30 08:22:25 -07:00
Connor McArthur
dc7ad2afc7
Merge pull request #1264 from fishtown-analytics/limit-exception-context
...
limit exception context scope
2019-01-29 13:33:00 -05:00
Connor McArthur
7e8ea51c1a
remove timezone from datetime properties
2019-01-29 11:30:08 -05:00
Connor McArthur
343afc2374
typo: switch datetime in for pytz
2019-01-29 11:18:04 -05:00
Jacob Beck
c2a0a2092a
source: now allows full-source selection, corresponding tests
2019-01-29 08:52:01 -07:00
Connor McArthur
f74a252b95
whitelist things from pytz, datetime
2019-01-29 10:50:18 -05:00
Connor McArthur
f5cfadae67
fix context exports for dbt.exceptions
2019-01-29 10:39:58 -05:00
Connor McArthur
7c1ecaf2b8
limit exception context scope
2019-01-29 09:44:58 -05:00
Jacob Beck
ea5edf20ba
Add source selector support
2019-01-28 16:53:42 -07:00
Jacob Beck
c5f8cc7816
Merge pull request #1258 from fishtown-analytics/fix/dont-create-unused-schemas
...
Only create schemas for selected nodes (#1239 )
2019-01-28 08:46:03 -07:00
Jacob Beck
60d75d26f0
Merge pull request #1260 from fishtown-analytics/fix/dbt-deps-foreign-langauges
...
Fix dbt deps on non-english shells (#1222 )
2019-01-28 08:16:20 -07:00
Jacob Beck
f6402d3390
Merge pull request #1254 from fishtown-analytics/feature/source-tables
...
Add 'sources' to dbt (#814 )
2019-01-28 08:05:22 -07:00
Jacob Beck
7b23a1b9a8
Make loader actually optional
...
Default to the empty string in ParsedSourceDefinitions
2019-01-25 07:37:09 -07:00
Jacob Beck
7714d12f7c
fix "git clone" on windows
...
env replaces, it does not update, and windows really needs its env
2019-01-24 16:56:38 -07:00
Jacob Beck
fc813e40eb
fix an old merge conflict
2019-01-24 15:36:50 -07:00
Jacob Beck
96578c3d2f
add support for "env" parameter to system.run_cmd, make every git command that parses output use it
2019-01-24 15:36:50 -07:00
Jacob Beck
f47be0808f
PR feedback
...
Add a test for model -> source
Cleaned up extra TODOs
Fixed missing variable in source_target_not_found
Changed a ref_target_not_found -> source_target_not_found
Better error messaging on invalid schema.yml files
Made loader optional
Make get_model_name_or_none accept just about anything
2019-01-24 15:25:35 -07:00
Drew Banin
b6d1e15a9f
Merge pull request #1251 from fishtown-analytics/fix/redshift-iam-auth-0.13.0
...
fix for optional adapter configs with aliases
2019-01-24 10:44:47 -05:00
Jacob Beck
7d332aaa35
fix snowflake check_schema_exists macro
2019-01-24 06:20:26 -07:00
Jacob Beck
9ff8705cd7
fix an old merge conflict
2019-01-23 16:35:03 -07:00
Jacob Beck
76669995f6
Only create schemas for selected models (and the default schema)
...
- pass selected uids along to the before_run method for schema creation
- only schemas used by selected nodes are created
- fix some big issues with check_schema_exists (such as: it does not work)
- integration test changes to test this
2019-01-23 16:35:03 -07:00
Jacob Beck
1079e9bfaf
test enhancements, make failure to find sources clearer
2019-01-23 14:11:03 -07:00
Jacob Beck
67d85316ac
make the test a bit sillier
2019-01-23 13:47:34 -07:00
Jacob Beck
7d41f4e22c
contracts test, squash a nodes iteration bug
2019-01-23 13:37:32 -07:00
Jacob Beck
cdeb0d1423
Merge pull request #1246 from fishtown-analytics/feature/presto-notimplemented-error
...
presto notimplemented errors (#1228 )
2019-01-23 11:18:36 -07:00
Jacob Beck
477699a102
Merge pull request #1245 from fishtown-analytics/feature/presto-adapter
...
Presto Adapter (#1106 , #1229 , #1230 )
2019-01-23 11:18:17 -07:00
Jacob Beck
63047d01ab
add a very basic integration test for sources
2019-01-23 09:48:01 -07:00
Jacob Beck
5e53e64df2
Add source configuration feature
...
- newly supported 'sources' section in schema.yml
- like models - define tests, columns, column tests, descriptions
- reference them via "source()" function (complements "ref()")
- new 'sources' field on nodes (like 'refs')
- many many contract and test updates to support it
- fixed a long-standing bug with referencing tests by namespace
- fix linecache on python 2.7
- rendering now forces strings into their native format to avoid "u" prefixes in tests
- fixup anything that iterates over nodes in the graph to accept ParsedSourceDefinitions
2019-01-23 09:48:01 -07:00
Connor McArthur
89207155fd
update RELEASE process
2019-01-21 13:59:37 -05:00
Drew Banin
b7d9eecf86
Merge pull request #1232 from alexyer/feature/snowflake-ssh-login
...
Add support for Snowflake Key Pair Authentication
2019-01-21 11:29:18 -05:00
Drew Banin
8212994018
push adapter-specific configs to plugins, add transient config
2019-01-19 23:37:03 -05:00
Drew Banin
c283cb0ff4
fix for optional configs with aliases
2019-01-19 15:30:59 -05:00
Drew Banin
a34ab9a268
Merge pull request #1250 from fishtown-analytics/s-sinter-dbt-cloud-g
...
Update README.md
2019-01-19 15:08:06 -05:00
Drew Banin
2a2a2b26ef
Update README.md
2019-01-19 15:07:24 -05:00
Jacob Beck
49fe2c3bb3
add NotImplemented archive implementation
2019-01-17 09:57:18 -07:00
Jacob Beck
170942c8be
add presto tests for archives + incremental failing
2019-01-17 09:57:06 -07:00
Alexander Yermakov
438b3529ae
Add support for Snowflake Key Pair Authentication
...
This PR adds support for Snowflake Key Pair Authentication.
Unit tests verify everything's getting passed through correctly.
2019-01-17 11:08:11 +02:00
Connor McArthur
2bff901860
pep8
2019-01-16 21:29:25 -05:00
Connor McArthur
d45fff3c5a
move comment with relevant code
2019-01-16 21:24:12 -05:00
Connor McArthur
8843a22854
undo changes to tox.ini
2019-01-16 21:22:54 -05:00
Connor McArthur
d9ba73af44
Merge branch 'dev/stephen-girard' of github.com:fishtown-analytics/dbt into add-node-level-timing-info
2019-01-16 21:21:10 -05:00
Jacob Beck
491d5935cf
make cancellation work, pretty ugly and no clue how to test it
2019-01-16 16:47:48 -07:00
Jacob Beck
85389afb3e
Presto testo
2019-01-16 16:47:48 -07:00
Jacob Beck
16d75249c5
Implement the presto adapter
...
- seeds, views, tables, and ephemeral models all implemented
- ConnectionManager methods and credentials
- Adapter.date_function
- macros
- give presto a small chunk size for seeds
- manually cascade drop_schema
- stub out some stuff that does not exist in presto
- stub out incremental materializations for now (delete is not useful)
- stub out query cancellation for now
2019-01-16 16:47:48 -07:00
Jacob Beck
27842f4cff
ran the create script
2019-01-16 16:47:48 -07:00
Jacob Beck
016afd4b2c
more work on the create script
2019-01-16 16:47:48 -07:00
Jacob Beck
cdb0cbdca7
Some dbt-core fixes/changes to support the new adapter
...
- fix sql docstring, which is just wrong.
- extract the default seed materialization to one that takes a chunk size param
- add NUMBERS constant for all number types dbt should support inserting
- update Column definition to allow for "varchar"
2019-01-16 16:47:43 -07:00
Jacob Beck
da2d7ea8c0
Create a docker-compose container system for presto testing
2019-01-16 16:46:56 -07:00
Jacob Beck
f6278d590a
initial work on an adapter building script
2019-01-16 16:46:56 -07:00
Jacob Beck
3b0d14bd5d
Merge pull request #1202 from fishtown-analytics/feature/dynamic-database-config
...
dynamic database config (#1183 , #1204 )
2019-01-16 16:46:33 -07:00
Connor McArthur
66d1f2099b
Merge branch 'dev/stephen-girard' of github.com:fishtown-analytics/dbt into add-node-level-timing-info
2019-01-16 16:45:31 -05:00
Jacob Beck
1ae32c12ab
Merge pull request #1226 from fishtown-analytics/fix/catch-everything
...
On unexpected errors in safe_run, do not raise (#1223 )
2019-01-16 14:44:22 -07:00
Connor McArthur
c626de76ff
fix test_events
2019-01-16 16:38:54 -05:00
Jacob Beck
c80792d713
Merge pull request #1207 from fishtown-analytics/feature/macros-for-everything
...
Convert all embedded adapter SQL into macros (#1204 )
2019-01-16 14:33:02 -07:00
Connor McArthur
a16958e35d
fix test_docs_generate integration test
2019-01-16 15:47:49 -05:00
Connor McArthur
3e4c75e41b
add timing as a namedproperty of RunModelResult
2019-01-16 15:22:15 -05:00
Jacob Beck
1596174a36
give root grant, split up test
2019-01-16 12:49:39 -07:00
Jacob Beck
f4084f069a
PR feedback from #1202
...
Postgres/Redshift permissions issues:
- use pg_views/pg_tables to list relations
- use pg_namespace to list schemas and check schema existence
Catalog:
- Redshift: fix error, add database check
- Snowflake: Use the information_schema_name macro
Snowflake/default: Quote the database properly in information_schema_name
2019-01-16 12:36:06 -07:00
Connor McArthur
aceee680c8
add thread id to run results
2019-01-16 12:40:08 -05:00
Connor McArthur
48c47bf11e
clean up, fix tests
2019-01-16 12:21:38 -05:00
Connor McArthur
8783c013e5
add node level timing info to run_results.json and to tracking
2019-01-15 20:41:08 -05:00
Drew Banin
aaa0127354
Merge pull request #1241 from fishtown-analytics/0.12.latest
...
Merge 0.12.latest into master
2019-01-15 17:01:16 -05:00
Jacob Beck
70069f53b1
Move SQL previously embedded into adapters into macros
...
Adapters now store an internal manifest that only has the dbt internal projects
Adapters use that manifest if none is provided to execute_manifest
The internal manifest is lazy-loaded to avoid recursion issues
Moved declared plugin paths down one level
Connection management changes to accomadate calling macro -> adapter -> macro
Split up precision and scale when describing number columns so agate doesn't eat commas
Manifest building now happens in the RunManager instead of the compiler
Now macros:
create/drop schema
get_columns_in_relation
alter column type
rename/drop/truncate
list_schemas/check_schema_exists
list_relations_without_caching
2019-01-15 08:08:45 -07:00
Jacob Beck
bf665e1c14
Merge branch 'dev/stephen-girard' into feature/dynamic-database-config
2019-01-15 07:58:35 -07:00
Connor McArthur
e359a69b18
Merge pull request #1224 from convoyinc/adriank/add_snowflake_sso_support
...
Add support for Snowflake SSO authentication round 2
2019-01-09 15:53:49 -05:00
Adrian Kreuziger
c01caefac9
Add support for Snowflake SSO authentication
2019-01-08 16:40:30 -08:00
Jacob Beck
2653201fe1
do not re-raise
2019-01-08 14:53:48 -07:00
Jacob Beck
dadab35aee
PR feedback
...
bigquery: naming/parameter sanity cleanup
postgres: never allow databases that aren't the default
postgres: simplify cache buliding since we know we'll only ever have one database
everything: parameter name change for execute_macro
everything: cache related bugfixes to casing
internal only: cross db/cross schema rename support in the cache
- none of the adapters support it, but unit tests expose the behavior
tests: much more comprehensive cache tests
2019-01-08 11:36:00 -07:00
Jacob Beck
c218af8512
Point at the 0.13 branch of dbt-utils
2019-01-08 11:36:00 -07:00
Jacob Beck
874ead9751
Make config() accept database, add adapter-specifc aliasing
...
Add concept of aliasing for credentials/relations
All databases use database, schema, and identifier internally now:
- Postgres/Redshift have 'dbname' aliased to database and pass to
password
- Bigquery has 'project' aliased to database and 'dataset' aliased to
schema
- Set default database include policy to True everywhere
config() calls accept aliases instead of canonical names
Remove unused functions and change others to accept Relations (see core/CHANGELOG.md)
Add catalog, etc support for multiple databases
2019-01-08 11:36:00 -07:00
Jacob Beck
1e5308db31
Merge remote-tracking branch 'origin/0.12.latest' into dev/stephen-girard
2019-01-08 11:32:20 -07:00
Connor McArthur
83c8381c19
Bump version: 0.12.2rc1 → 0.12.2
2019-01-07 14:14:41 -05:00
Connor McArthur
b5f5117555
Merge pull request #1220 from fishtown-analytics/dev/grace-kelly
...
Release: Grace Kelly (0.12.2)
2019-01-07 14:13:50 -05:00
Drew Banin
b0a4f5c981
Merge pull request #1216 from mikekaminsky/docs/dev-installation
...
Updates contributing doc with new local installation instructions
2019-01-07 13:25:14 -05:00
Drew Banin
1da50abe2f
Merge pull request #1200 from fishtown-analytics/changelog/grace-kelly
...
Update CHANGELOG.md for Grace Kelly
2019-01-07 11:06:02 -05:00
Drew Banin
6d69ff0bda
Update CHANGELOG.md
...
add links to docs
2019-01-07 11:04:39 -05:00
Jacob Beck
7179d135fa
Merge pull request #1218 from fishtown-analytics/fix/lets-not-hang-on-errors
...
move cleanup into the executing thread (#1214 )
2019-01-07 08:38:41 -07:00
Jacob Beck
3e4523e1ef
On errors inside the thread, set a raise_next_tick flag
...
Threads that raise exceptions bypass the callback, which makes dbt hang.
Now threads don't raise during the callback, instead they set a flag.
The RunManager will check the flag during queue processing and raise if set.
Fix compilation failures so they raise properly.
2019-01-07 07:58:23 -07:00
Michael Kaminsky
69a65dd63f
Updates contributing doc with new local installation instructions
2019-01-05 20:45:42 -06:00
Drew Banin
3c25a9b40d
Merge pull request #1210 from fishtown-analytics/fix/dbt-init-command
...
do not try to remove remote after dbt init
2019-01-04 14:56:28 -05:00
Drew Banin
f43d9b5e88
pr feedback
2019-01-04 13:14:11 -05:00
Drew Banin
5826bc80bc
( fixes #1209 ) do not try to remove remote after dbt init
2019-01-03 22:27:51 -05:00
Drew Banin
6563b03299
Merge pull request #1208 from fishtown-analytics/fix/repo-file-placement
...
move markdown files back into place
2019-01-03 18:13:53 -05:00
Drew Banin
406ff55c7d
move markdown files back into place
2019-01-03 18:03:46 -05:00
Adrian
8882bbe617
Merge pull request #1 from fishtown-analytics/dev/stephen-girard
...
Merge upstream
2019-01-03 13:09:05 -08:00
Drew Banin
769a886b93
Merge pull request #1206 from fishtown-analytics/fix/windows-unicode-output-for-dbt-debug
...
Replace unicode characters with ascii strings
2019-01-03 13:27:38 -05:00
Jacob Beck
f2fc002f5c
Merge pull request #1145 from fishtown-analytics/dbt-core
...
dbt core/plugin architecture (#1070 , #1069 )
2019-01-02 16:59:29 -07:00
Jacob Beck
836998c9e9
Merge branch 'dev/stephen-girard' into dbt-core
2019-01-02 11:30:31 -07:00
Drew Banin
f90a5b14ad
( fixes #1201 ) Replace unicode characters with ascii strings
2019-01-02 10:51:50 -05:00
Drew Banin
6004bdf012
Merge pull request #1191 from brianhartsock/archive_character_columns
...
Ensure character columns are treated as string types.
2018-12-21 13:22:26 -05:00
Drew Banin
d7610a7c55
Merge pull request #1192 from patrickgoss/fix/postgres-dependency-query
...
Postgresql dependency query speedup
2018-12-21 13:21:01 -05:00
Drew Banin
2ecc1e06cf
Merge pull request #1198 from vijaykiran/dev/grace-kelly
...
Print dbt version before every task is run
2018-12-21 13:16:08 -05:00
Drew Banin
dcc017d681
Merge pull request #1199 from vijaykiran/update-contributing-doc
...
Update environment setup in CONTRIBUTING.md
2018-12-21 13:14:36 -05:00
Drew Banin
ea401f6556
Update CHANGELOG.md
2018-12-21 13:07:19 -05:00
Vijay Kiran
09fbe288d8
Update environment setup in CONTRIBUTING.md
...
- Document command to activate the virtualenv
- Fix minor typo
2018-12-21 19:02:57 +01:00
Vijay Kiran
7786175d32
Print dbt version before every task is run
...
resolves fishtown-analytics/dbt#1134
2018-12-21 17:58:35 +01:00
Jacob Beck
f558516f40
remove extra script
2018-12-20 08:41:04 -07:00
Jacob Beck
d8c46d94df
fix the issue on bigquery where tests executed with a connection name of "master"
2018-12-20 08:40:13 -07:00
pgoss
f64e335735
split the relationship CTE in order to speed the dependency query on large dbs
2018-12-19 12:32:32 -05:00
Brian Hartsock
b263ba7df2
Ensure character columns are treated as string types.
2018-12-19 10:31:11 -05:00
Jacob Beck
d2a68d92a3
Merge pull request #1176 from fishtown-analytics/fix/remove-operations
...
Remove operations [#1117 ]
2018-12-18 12:23:46 -07:00
Jacob Beck
4cbff8e1a1
update changelog
2018-12-18 08:28:08 -07:00
Jacob Beck
33ffafc7d6
run_operation -> execute_macro
2018-12-18 08:01:59 -07:00
Jacob Beck
4780c4bb18
Split dbt into core and plugins
2018-12-14 11:17:41 -07:00
Jacob Beck
c61561aab2
Merge pull request #1178 from fishtown-analytics/feature/package-dbt-version-compatibility
...
package dbt version compatibility (#581 )
2018-12-14 10:15:19 -08:00
Jacob Beck
9466862560
add require-dbt-version config item and --no-version-check flag, make dependency errors fail the run
2018-12-14 10:33:05 -07:00
Jacob Beck
931dd4e301
Merge branch 'dev/grace-kelly' into dev/stephen-girard
2018-12-14 10:16:40 -07:00
Jacob Beck
e52475cac7
Merge pull request #1186 from fishtown-analytics/fix/tracking
...
make tracking use the profile directory, and suppress errors (#1180 )
2018-12-14 08:18:35 -08:00
Claire Carroll
afa9fc051e
Merge pull request #1189 from fishtown-analytics/add-m-flag-for-tests
...
Add -m argument for dbt test
2018-12-14 11:15:51 -05:00
Claire Carroll
3cbf49cba7
Add m flag for tests
2018-12-13 16:46:06 -05:00
Jacob Beck
5deb7e8c2d
everything about tracking is hard
2018-12-13 14:04:53 -07:00
Jacob Beck
2003222691
Merge branch 'dev/grace-kelly' into dev/stephen-girard
2018-12-13 12:58:44 -07:00
Jacob Beck
08913bf96b
Merge pull request #1188 from fishtown-analytics/ipdb-dev-requirement
...
Add ipdb to dev requirements so it shows up in test docker containers
2018-12-13 11:48:09 -08:00
Jacob Beck
15c047077a
Add ipdb to dev requirements so it shows up in test docker containers for debugging
2018-12-13 10:28:39 -07:00
Jacob Beck
260bcfd532
Merge branch 'dev/grace-kelly' into dev/stephen-girard
2018-12-13 09:07:16 -07:00
Claire Carroll
3deb295d29
Merge pull request #1147 from fishtown-analytics/cleanup-repo
...
Cleanup repo and update readme
2018-12-13 09:50:04 -05:00
Claire Carroll
f56ac542bc
Merge branch 'dev/grace-kelly' into cleanup-repo
2018-12-13 09:46:25 -05:00
Claire Carroll
ebd6d3ef19
Update readme and cleanup repo
2018-12-13 09:43:38 -05:00
Jacob Beck
808ed75858
make tracking use the profile directory, and suppress errors
2018-12-12 16:01:42 -07:00
Connor McArthur
d2c704884e
Bump version: 0.12.2a1 → 0.12.2rc1
2018-12-07 14:10:57 -05:00
Jacob Beck
91a5b1ce52
Merge branch 'dev/grace-kelly' into dev/stephen-girard
2018-12-05 12:14:54 -07:00
Jacob Beck
bec30efec5
Merge pull request #1174 from fishtown-analytics/feature/latest-version-from-pypi
...
use pypi for the latest version instead of git [#1122 ]
2018-12-05 11:56:44 -07:00
Drew Banin
2cd24cfa9e
Merge pull request #1129 from fishtown-analytics/feature/better-incremental-models
...
(fixes #744 ) deprecate sql_where and provide an alternative
2018-12-05 12:52:45 -05:00
Jacob Beck
963fb84cb7
Merge pull request #1171 from fishtown-analytics/feature/make-debug-great
...
Improve dbt debug [#1061 ]
2018-12-05 10:52:03 -07:00
Connor McArthur
65729c4acc
Bump version: 0.12.1 → 0.12.2a1
2018-12-05 12:15:50 -05:00
Jacob Beck
9b8e8ff17a
more pr feedback
2018-12-05 10:12:57 -07:00
Jacob Beck
8ea9c68be0
PR feedback
2018-12-05 10:00:30 -07:00
Jacob Beck
282774cbdf
Merge branch 'dev/grace-kelly' into dev/stephen-girard
2018-12-05 09:49:07 -07:00
Jacob Beck
eb504ae866
use pypi for the latest version instead of git
2018-12-05 09:29:30 -07:00
Jacob Beck
0f1520c392
re-add call to path_info
2018-12-05 09:23:11 -07:00
Drew Banin
009eaa3a59
pep8
2018-12-05 11:06:04 -05:00
Drew Banin
80232ff8e8
( fixes #744 ) deprecate sql_where and provide an alternative
2018-12-05 10:44:20 -05:00
Jacob Beck
9938af1580
pr feedback
2018-12-05 08:33:16 -07:00
Jacob Beck
6935a4a2e4
pr feedback: add pretty colors
2018-12-05 08:16:41 -07:00
Jacob Beck
eef5024354
Merge branch 'dev/grace-kelly' into feature/make-debug-great
2018-12-05 08:12:31 -07:00
Jacob Beck
bb6b469768
Merge pull request #1157 from fishtown-analytics/feature/graph-iteration
...
Handle concurrent job ordering via a graph iterator (#813 )
2018-12-05 08:10:11 -07:00
Jacob Beck
d4c2dfedb2
dbt debug
2018-12-05 07:34:55 -07:00
Jacob Beck
3434ad9ca0
Bypass project loading in init, we do it anyway
2018-12-05 07:34:55 -07:00
Jacob Beck
937219dd91
add connection_info method to credentials for dbt debug
2018-12-05 07:34:55 -07:00
Jacob Beck
5bdd1ebdbc
Merge pull request #1159 from fishtown-analytics/fix/use-root-generate-schema-name
...
get the default schema name generator from the root package [#801 ]
2018-12-05 07:32:52 -07:00
Jacob Beck
17f3f24652
Merge pull request #1168 from fishtown-analytics/fix/snowflake-drops-wrong-backup-type
...
drop the correct relation type on snowflake when building tables [#1103 ]
2018-12-05 07:32:25 -07:00
Jacob Beck
d80b37854a
Merge pull request #1169 from fishtown-analytics/feature/log-warnings-on-missing-test-refs
...
log warnings on missing test refs [#968 ]
2018-12-05 07:32:02 -07:00
Jacob Beck
cbfa21ce45
PR feedback
2018-12-04 15:04:10 -07:00
Jacob Beck
3665e65986
reorganize some things to make us better detect disabled vs missing nodes
2018-12-04 11:16:46 -07:00
Jacob Beck
0daca0276b
log about missing test ref targets at warning instead of debug level, include filepath
2018-12-04 09:33:24 -07:00
Jacob Beck
8769118471
drop the correct relation type on snowflake when building tables
2018-12-04 09:09:32 -07:00
Jacob Beck
863dbd2f4d
Merge branch 'dev/grace-kelly' into fix/use-root-generate-schema-name
2018-12-03 16:36:46 -07:00
Jacob Beck
eb00b1a1b9
Merge pull request #1166 from fishtown-analytics/fix/trim-schema-whitespace
...
Trim schema whitespace (#1074 )
2018-12-03 16:35:27 -07:00
Jacob Beck
953ba9b8eb
Merge pull request #1163 from fishtown-analytics/fix/cdecimal-json-encoding
...
handle cdecimal.Decimal during serialization [#1155 ]
2018-12-03 16:29:53 -07:00
Jacob Beck
aa9d43a3fc
Merge pull request #1164 from fishtown-analytics/fix/analysis-parsing-inside-raw
...
fix analysis to only compile sql once like model nodes [#1152 ]
2018-12-03 16:29:31 -07:00
Jacob Beck
9d5cbf7e51
strip the schema when there is extra whitespace
2018-12-03 14:35:32 -07:00
Jacob Beck
1744f21084
Refactor nodes into objects, make generate_schema_name use its own context
...
Allows us to cache the get_schema behavior without re-building it for every node
Allows us to special-case the generate_schema_name macro
2018-12-03 14:10:59 -07:00
Jacob Beck
adf05bd11d
get the default schema name generator from the root package
2018-12-03 14:09:57 -07:00
Jacob Beck
3d205c3597
fix analysis to only compile sql once like model nodes, fix test
2018-12-03 13:06:52 -07:00
Jacob Beck
9f9b861769
handle cdecimal.Decimal during serialization as well as decimal.Decimal
2018-12-03 07:48:51 -07:00
Jacob Beck
6025d3d843
Merge pull request #1161 from fishtown-analytics/feature/m-is-for-models
...
add "-m" shorthand for models (#1160 )
2018-11-30 15:33:12 -07:00
Jacob Beck
3cf03f3018
add "-m" shorthand for models
2018-11-30 15:25:29 -07:00
Jacob Beck
1c0caf9a81
Merge branch 'dev/grace-kelly' into dev/stephen-girard
2018-11-29 14:24:04 -07:00
Jacob Beck
4dc79f655f
pr feedback, add many docstrings
2018-11-29 08:04:23 -07:00
Jacob Beck
5a06d57d7e
use the manifest for node information and block/timeout arguments to get()
...
Assorted cleanup/test fixes
2018-11-28 15:14:45 -07:00
Jacob Beck
84ba7f57d0
fix many bugs, move RunBuilder back into the RunManager
2018-11-28 14:14:00 -07:00
Connor McArthur
8af30611f3
Merge pull request #1133 from fishtown-analytics/bigquery-smarter-delete-dataset
...
Bigquery: smarter delete_dataset
2018-11-28 09:40:14 -05:00
Jacob Beck
6e27476faa
Merge pull request #1105 from fishtown-analytics/fix/hub-packaging
...
Fix the hub packaging so that it at least mostly works
2018-11-28 07:39:19 -07:00
Jacob Beck
acddb3b939
RunManager now operates on queues instead of lists
2018-11-27 15:26:16 -07:00
Jacob Beck
b6193be1ef
add initial work on a graph-based priority queue
2018-11-27 15:26:16 -07:00
Jacob Beck
e7b1a093a3
Refactor the RunManager to build its runners as it iterates over the nodes instead of in advance
2018-11-27 15:26:16 -07:00
Jacob Beck
5be8c7f85f
Merge pull request #1150 from fishtown-analytics/feature/config-muliple-calls
...
allow muliple config() calls (#558 )
2018-11-26 12:16:06 -07:00
Jacob Beck
b751ed6c6a
Merge pull request #1146 from fishtown-analytics/feature/hook-config-aliases
...
add pre_hook/post_hook aliases to config (#1124 )
2018-11-26 11:55:22 -07:00
Jacob Beck
d16ca86782
Merge pull request #1151 from fishtown-analytics/feature/bq-profile-location
...
Add 'location' to google bigquery profile (#969 )
2018-11-26 11:54:20 -07:00
Jacob Beck
b92d6692ce
test that location gets passed along
2018-11-23 13:09:12 -07:00
Jacob Beck
dab2ff402f
add a location field to the bigquery profile and pass it along to the google library if it is provided
2018-11-23 11:04:46 -07:00
Jacob Beck
51252b06b9
Add a test for overwrite/append
2018-11-23 10:12:24 -07:00
Jacob Beck
1fd84ad9d5
add unit tests
2018-11-23 10:12:24 -07:00
Jacob Beck
c4d6b2ed0f
Delete dead code, move SourceConfig into the parser, allow multiple calls
2018-11-23 10:12:24 -07:00
Jacob Beck
71a239825a
Merge branch 'dev/grace-kelly' into feature/hook-config-aliases
2018-11-23 08:11:31 -07:00
Jacob Beck
f72e0a8ddf
Merge pull request #1148 from fishtown-analytics/feature/jinja-do-statement
...
Jinja expression statements (#1113 )
2018-11-23 06:38:18 -07:00
Jacob Beck
069bc3a905
add do-syntax
2018-11-21 14:53:13 -07:00
Jacob Beck
0307d78236
PR feedback, tests around it
2018-11-21 12:29:19 -07:00
Jacob Beck
e543dc4278
add pre_hook/post_hook kwargs to config, add tests
2018-11-21 11:54:45 -07:00
Jacob Beck
029ef1795f
Merge pull request #1020 from fishtown-analytics/profiler
...
dbt builtin profiler
2018-11-20 07:57:44 -07:00
Jacob Beck
12433fdba4
Merge branch 'dev/grace-kelly' into profiler
2018-11-20 07:08:49 -07:00
Jacob Beck
0a66adf707
Merge pull request #1142 from fishtown-analytics/feature/profiler-with-single-threaded
...
single-threaded mode
2018-11-19 19:06:30 -07:00
Jacob Beck
b5aab26c38
Merge pull request #1144 from fishtown-analytics/fix/run-repeatability-issues
...
Fix run repeatability/caching issues (#1138 , #1139 , #1140 )
2018-11-19 19:04:44 -07:00
Drew Banin
416173a03c
Merge pull request #1128 from fishtown-analytics/cache-macro-only-manifest
...
cache the macro manifest
2018-11-19 20:47:53 -05:00
Jacob Beck
e82361c893
lowercase the cache's view of schema/identifier, fix drop
2018-11-19 09:51:27 -07:00
Jacob Beck
7d3bf03404
fix casing expectations in unit tests
2018-11-19 09:51:27 -07:00
Jacob Beck
eb50b8319b
Add integration tests with interesting model/case/quoting settings
2018-11-19 09:51:27 -07:00
Jacob Beck
cfd2d60575
stop dropping renames...
2018-11-19 09:51:27 -07:00
Jacob Beck
d4c3fb8261
fix schema cache casing bugs
2018-11-19 09:51:27 -07:00
Jacob Beck
7940b71ffe
on dbt invocation, reset the adapters (to reset the cache)
2018-11-19 09:51:27 -07:00
Jacob Beck
6dd04b1a43
set cache logger propagation during invocation time instead of at log init time
2018-11-19 09:51:25 -07:00
Jacob Beck
399a6854c5
add a special flag to make dbt use map instead of the pool
2018-11-19 09:21:59 -07:00
Drew Banin
8eded7081c
PR feedback (move comment)
2018-11-19 10:38:12 -05:00
Drew Banin
3bdebba18d
Merge pull request #1123 from fishtown-analytics/fix/local-packages
...
fix for deps that depend on other local deps
2018-11-19 10:17:56 -05:00
Jacob Beck
8aab340a2a
Merge branch 'dev/grace-kelly' into profiler
2018-11-19 08:13:34 -07:00
Drew Banin
0138228309
Merge branch '0.12.latest' into dev/grace-kelly
2018-11-15 16:05:58 -05:00
Drew Banin
e60280c4d6
Merge branch '0.12.latest'
2018-11-15 12:24:05 -05:00
Connor McArthur
3912028318
bigquery: use delete_contents option in delete_dataset, remove unused drop_tables_in_schema
2018-11-15 10:45:04 -05:00
Connor McArthur
59cea11ef5
Bump version: 0.12.0 → 0.12.1
2018-11-15 09:28:22 -05:00
Connor McArthur
d9c12abd2d
Merge pull request #1131 from fishtown-analytics/dev/0.12.1
...
dbt 0.12.1
2018-11-15 09:27:34 -05:00
Drew Banin
4b981caa53
first cut at caching macro manifest
2018-11-14 21:33:16 -05:00
Jacob Beck
735ff8831d
Merge pull request #1110 from fishtown-analytics/fix/dbt-deps-issues
...
Fix dbt deps issues (#778 #994 #895 )
2018-11-14 13:31:28 -07:00
Jacob Beck
6529c3edd3
Merge pull request #1109 from fishtown-analytics/feature/adapter-refactor
...
Feature/adapter refactor (#962 , #963 , #965 , #1035 , #1090 )
2018-11-14 11:32:05 -07:00
Jacob Beck
8840996a30
user feedback: log download directory in dbt deps
2018-11-14 11:30:17 -07:00
Jacob Beck
d35e549dbf
Handle cross-drive windows permissions issues by undoing git's readonly settings
2018-11-14 10:37:23 -07:00
Drew Banin
7195f07b3d
fix for local deps that depend on other local deps
2018-11-13 21:55:25 -05:00
Jacob Beck
351542257e
Merge branch 'dev/grace-kelly' into fix/dbt-deps-issues
2018-11-13 11:32:42 -07:00
Jacob Beck
8927aa8e02
clean up TODOs
2018-11-13 09:54:10 -07:00
Drew Banin
aef7866e29
Update CHANGELOG.md
2018-11-13 10:36:35 -05:00
Drew Banin
70694e3bb9
Merge pull request #1118 from fishtown-analytics/0.12.latest
...
merge 0.12.latest to master
2018-11-13 10:19:56 -05:00
Jacob Beck
717d1ed995
Merge pull request #1107 from fishtown-analytics/no-postgres-for-non-postgres
...
On non-postgres tests, don't require the postgres container (#841 )
2018-11-13 08:02:28 -07:00
Jacob Beck
3773843094
in deps, when "git" is not in the path, link users to help docs
2018-11-13 08:02:08 -07:00
Jacob Beck
9bee0190d2
Lazy load adapters
...
Move adapter package loading to runtime, after configs have been mostly parsed
Parse/validate credentials after determining what the type is
profile/config contracts ignore credentials
2018-11-12 11:30:28 -07:00
Jacob Beck
60c4619862
Make adapters optional
2018-11-12 11:30:28 -07:00
Jacob Beck
9ffbb3ad02
Split out connection managers
2018-11-12 11:30:28 -07:00
Jacob Beck
350b81db99
Class hierarchy, deprecate and remove deprecated methods, abstract method definitions
2018-11-12 11:30:28 -07:00
Jacob Beck
412b165dc9
fix issue where we incorrectly logged stack traces
2018-11-12 11:29:45 -07:00
Jacob Beck
531d7c687e
use environment variables or per-run temp directories to assign the temporary downloads directory
2018-11-12 11:29:45 -07:00
Jacob Beck
e866caa900
add support for cross-drive moves
2018-11-12 11:29:45 -07:00
Jacob Beck
ec466067b2
Tests, fixup ci
2018-11-12 11:29:45 -07:00
Jacob Beck
59b6f78c71
Raise an exception on rc!=0 in run_cmd, raise more specific exceptions about what went wrong on error
2018-11-12 11:29:45 -07:00
Jacob Beck
7757c85d4f
add some helpful exceptions about command running
2018-11-12 11:29:45 -07:00
Jacob Beck
3077eecb97
Avoid connecting to postgres on each run
2018-11-12 11:29:11 -07:00
Jacob Beck
e6fc0f6724
make 2.7 exceptions have the same output as 3.x
2018-11-12 11:27:43 -07:00
Jacob Beck
42f817abf5
contracts and very basic tests for hub packages
2018-11-12 11:27:43 -07:00
Connor McArthur
3b0c9f8b48
merged dev/gb
2018-10-29 10:55:33 -04:00
Connor McArthur
738304f438
dbt builtin profiler
2018-09-20 08:44:42 -04:00