mirror of
https://github.com/dbt-labs/dbt-project-evaluator.git
synced 2025-12-18 02:11:27 +00:00
1340 lines
103 KiB
Plaintext
1340 lines
103 KiB
Plaintext
2022-01-07 16:46:02.968597 (MainThread): Running with dbt=0.18.1
|
|
2022-01-07 16:46:03.295892 (MainThread): Loading KWallet
|
|
2022-01-07 16:46:03.297293 (MainThread): Loading SecretService
|
|
2022-01-07 16:46:03.298333 (MainThread): Loading Windows
|
|
2022-01-07 16:46:03.299920 (MainThread): Loading chainer
|
|
2022-01-07 16:46:03.300746 (MainThread): Loading macOS
|
|
2022-01-07 16:46:04.039265 (MainThread): running dbt with arguments Namespace(cls=<class 'dbt.task.generate.GenerateTask'>, compile=True, debug=False, defer=None, exclude=None, log_cache_events=False, log_format='default', models=None, partial_parse=None, profile=None, profiles_dir='/Users/daveconnors/.dbt', project_dir=None, record_timing_info=None, rpc_method='docs.generate', selector_name=None, single_threaded=False, state=None, strict=False, target=None, test_new_parser=False, threads=None, use_cache=True, use_colors=None, vars='{}', version_check=True, warn_error=False, which='generate', write_json=True)
|
|
2022-01-07 16:46:04.042849 (MainThread): Tracking: tracking
|
|
2022-01-07 16:46:04.043361 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'start', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x104e5c370>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x104f14820>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x104f147c0>]}
|
|
2022-01-07 16:46:04.080679 (MainThread): Partial parsing not enabled
|
|
2022-01-07 16:46:04.083046 (MainThread): Parsing macros/catalog.sql
|
|
2022-01-07 16:46:04.086399 (MainThread): Parsing macros/adapters.sql
|
|
2022-01-07 16:46:04.124883 (MainThread): Parsing macros/materializations/merge.sql
|
|
2022-01-07 16:46:04.127656 (MainThread): Parsing macros/materializations/view.sql
|
|
2022-01-07 16:46:04.129594 (MainThread): Parsing macros/materializations/table.sql
|
|
2022-01-07 16:46:04.134483 (MainThread): Parsing macros/materializations/incremental.sql
|
|
2022-01-07 16:46:04.150297 (MainThread): Parsing macros/core.sql
|
|
2022-01-07 16:46:04.155533 (MainThread): Parsing macros/materializations/helpers.sql
|
|
2022-01-07 16:46:04.166538 (MainThread): Parsing macros/materializations/snapshot/snapshot_merge.sql
|
|
2022-01-07 16:46:04.168819 (MainThread): Parsing macros/materializations/snapshot/strategies.sql
|
|
2022-01-07 16:46:04.191470 (MainThread): Parsing macros/materializations/snapshot/snapshot.sql
|
|
2022-01-07 16:46:04.227034 (MainThread): Parsing macros/materializations/seed/seed.sql
|
|
2022-01-07 16:46:04.253808 (MainThread): Parsing macros/materializations/incremental/helpers.sql
|
|
2022-01-07 16:46:04.256262 (MainThread): Parsing macros/materializations/incremental/incremental.sql
|
|
2022-01-07 16:46:04.264591 (MainThread): Parsing macros/materializations/common/merge.sql
|
|
2022-01-07 16:46:04.282595 (MainThread): Parsing macros/materializations/table/table.sql
|
|
2022-01-07 16:46:04.291318 (MainThread): Parsing macros/materializations/view/view.sql
|
|
2022-01-07 16:46:04.298875 (MainThread): Parsing macros/materializations/view/create_or_replace_view.sql
|
|
2022-01-07 16:46:04.305918 (MainThread): Parsing macros/etc/get_custom_alias.sql
|
|
2022-01-07 16:46:04.307544 (MainThread): Parsing macros/etc/query.sql
|
|
2022-01-07 16:46:04.309151 (MainThread): Parsing macros/etc/is_incremental.sql
|
|
2022-01-07 16:46:04.311400 (MainThread): Parsing macros/etc/datetime.sql
|
|
2022-01-07 16:46:04.323783 (MainThread): Parsing macros/etc/get_custom_schema.sql
|
|
2022-01-07 16:46:04.326544 (MainThread): Parsing macros/etc/get_custom_database.sql
|
|
2022-01-07 16:46:04.328729 (MainThread): Parsing macros/adapters/common.sql
|
|
2022-01-07 16:46:04.388593 (MainThread): Parsing macros/schema_tests/relationships.sql
|
|
2022-01-07 16:46:04.391514 (MainThread): Parsing macros/schema_tests/not_null.sql
|
|
2022-01-07 16:46:04.393888 (MainThread): Parsing macros/schema_tests/unique.sql
|
|
2022-01-07 16:46:04.396495 (MainThread): Parsing macros/schema_tests/accepted_values.sql
|
|
2022-01-07 16:46:04.411274 (MainThread): Partial parsing not enabled
|
|
2022-01-07 16:46:04.479880 (MainThread): Acquiring new snowflake connection "model.my_new_project.stg_model_3".
|
|
2022-01-07 16:46:04.509899 (MainThread): Acquiring new snowflake connection "model.my_new_project.stg_model_2".
|
|
2022-01-07 16:46:04.527512 (MainThread): Acquiring new snowflake connection "model.my_new_project.stg_model_1".
|
|
2022-01-07 16:46:04.543637 (MainThread): Acquiring new snowflake connection "model.my_new_project.fct_model_6".
|
|
2022-01-07 16:46:04.561731 (MainThread): Acquiring new snowflake connection "model.my_new_project.dim_model_7".
|
|
2022-01-07 16:46:04.574485 (MainThread): Acquiring new snowflake connection "model.my_new_project.int_model_4".
|
|
2022-01-07 16:46:04.591609 (MainThread): Acquiring new snowflake connection "model.my_new_project.int_model_5".
|
|
2022-01-07 16:46:04.732570 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'end', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x105e3bc10>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x105e44fd0>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x105e44fa0>]}
|
|
2022-01-07 16:46:04.732893 (MainThread): Flushing usage events
|
|
2022-01-07 16:46:05.212289 (MainThread): Connection 'model.my_new_project.int_model_5' was properly closed.
|
|
2022-01-07 16:46:05.212539 (MainThread): Encountered an error:
|
|
2022-01-07 16:46:05.212730 (MainThread): Compilation Error in model stg_model_3 (models/staging/stg_model_3.sql)
|
|
Model 'model.my_new_project.stg_model_3' (models/staging/stg_model_3.sql) depends on a source named 'source_3.table_1' which was not found
|
|
2022-01-07 16:46:05.217395 (MainThread): Traceback (most recent call last):
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/main.py", line 124, in main
|
|
results, succeeded = handle_and_check(args)
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/main.py", line 202, in handle_and_check
|
|
task, res = run_from_args(parsed)
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/main.py", line 255, in run_from_args
|
|
results = task.run()
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/task/generate.py", line 213, in run
|
|
compile_results = CompileTask.run(self)
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/task/runnable.py", line 399, in run
|
|
self._runtime_initialize()
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/task/runnable.py", line 118, in _runtime_initialize
|
|
super()._runtime_initialize()
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/task/runnable.py", line 78, in _runtime_initialize
|
|
self.load_manifest()
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/task/runnable.py", line 65, in load_manifest
|
|
self.manifest = get_full_manifest(self.config)
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/perf_utils.py", line 28, in get_full_manifest
|
|
return load_manifest(
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/parser/manifest.py", line 748, in load_manifest
|
|
return ManifestLoader.load_all(config, macro_manifest, macro_hook)
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/parser/manifest.py", line 351, in load_all
|
|
manifest = loader.create_manifest()
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/parser/manifest.py", line 327, in create_manifest
|
|
self.process_manifest(manifest)
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/parser/manifest.py", line 297, in process_manifest
|
|
process_sources(manifest, project_name)
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/parser/manifest.py", line 706, in process_sources
|
|
_process_sources_for_node(manifest, current_project, node)
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/parser/manifest.py", line 689, in _process_sources_for_node
|
|
invalid_source_fail_unless_test(
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/parser/manifest.py", line 406, in invalid_source_fail_unless_test
|
|
source_target_not_found(
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/exceptions.py", line 569, in source_target_not_found
|
|
raise_compiler_error(msg, model)
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/exceptions.py", line 407, in raise_compiler_error
|
|
raise CompilationException(msg, node)
|
|
dbt.exceptions.CompilationException: Compilation Error in model stg_model_3 (models/staging/stg_model_3.sql)
|
|
Model 'model.my_new_project.stg_model_3' (models/staging/stg_model_3.sql) depends on a source named 'source_3.table_1' which was not found
|
|
|
|
2022-01-07 16:46:35.512882 (MainThread): Running with dbt=0.18.1
|
|
2022-01-07 16:46:35.747521 (MainThread): Loading KWallet
|
|
2022-01-07 16:46:35.748962 (MainThread): Loading SecretService
|
|
2022-01-07 16:46:35.750396 (MainThread): Loading Windows
|
|
2022-01-07 16:46:35.751777 (MainThread): Loading chainer
|
|
2022-01-07 16:46:35.752542 (MainThread): Loading macOS
|
|
2022-01-07 16:46:36.445119 (MainThread): running dbt with arguments Namespace(cls=<class 'dbt.task.generate.GenerateTask'>, compile=True, debug=False, defer=None, exclude=None, log_cache_events=False, log_format='default', models=None, partial_parse=None, profile=None, profiles_dir='/Users/daveconnors/.dbt', project_dir=None, record_timing_info=None, rpc_method='docs.generate', selector_name=None, single_threaded=False, state=None, strict=False, target=None, test_new_parser=False, threads=None, use_cache=True, use_colors=None, vars='{}', version_check=True, warn_error=False, which='generate', write_json=True)
|
|
2022-01-07 16:46:36.448177 (MainThread): Tracking: tracking
|
|
2022-01-07 16:46:36.448838 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'start', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x106e4a2e0>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x106f027f0>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x106f02790>]}
|
|
2022-01-07 16:46:36.484415 (MainThread): Partial parsing not enabled
|
|
2022-01-07 16:46:36.486337 (MainThread): Parsing macros/catalog.sql
|
|
2022-01-07 16:46:36.489999 (MainThread): Parsing macros/adapters.sql
|
|
2022-01-07 16:46:36.536724 (MainThread): Parsing macros/materializations/merge.sql
|
|
2022-01-07 16:46:36.539621 (MainThread): Parsing macros/materializations/view.sql
|
|
2022-01-07 16:46:36.541630 (MainThread): Parsing macros/materializations/table.sql
|
|
2022-01-07 16:46:36.546546 (MainThread): Parsing macros/materializations/incremental.sql
|
|
2022-01-07 16:46:36.566256 (MainThread): Parsing macros/core.sql
|
|
2022-01-07 16:46:36.571723 (MainThread): Parsing macros/materializations/helpers.sql
|
|
2022-01-07 16:46:36.584538 (MainThread): Parsing macros/materializations/snapshot/snapshot_merge.sql
|
|
2022-01-07 16:46:36.586817 (MainThread): Parsing macros/materializations/snapshot/strategies.sql
|
|
2022-01-07 16:46:36.613665 (MainThread): Parsing macros/materializations/snapshot/snapshot.sql
|
|
2022-01-07 16:46:36.653595 (MainThread): Parsing macros/materializations/seed/seed.sql
|
|
2022-01-07 16:46:36.683296 (MainThread): Parsing macros/materializations/incremental/helpers.sql
|
|
2022-01-07 16:46:36.685642 (MainThread): Parsing macros/materializations/incremental/incremental.sql
|
|
2022-01-07 16:46:36.694500 (MainThread): Parsing macros/materializations/common/merge.sql
|
|
2022-01-07 16:46:36.717379 (MainThread): Parsing macros/materializations/table/table.sql
|
|
2022-01-07 16:46:36.727088 (MainThread): Parsing macros/materializations/view/view.sql
|
|
2022-01-07 16:46:36.735145 (MainThread): Parsing macros/materializations/view/create_or_replace_view.sql
|
|
2022-01-07 16:46:36.741685 (MainThread): Parsing macros/etc/get_custom_alias.sql
|
|
2022-01-07 16:46:36.743311 (MainThread): Parsing macros/etc/query.sql
|
|
2022-01-07 16:46:36.744987 (MainThread): Parsing macros/etc/is_incremental.sql
|
|
2022-01-07 16:46:36.747311 (MainThread): Parsing macros/etc/datetime.sql
|
|
2022-01-07 16:46:36.758501 (MainThread): Parsing macros/etc/get_custom_schema.sql
|
|
2022-01-07 16:46:36.761436 (MainThread): Parsing macros/etc/get_custom_database.sql
|
|
2022-01-07 16:46:36.763876 (MainThread): Parsing macros/adapters/common.sql
|
|
2022-01-07 16:46:36.816830 (MainThread): Parsing macros/schema_tests/relationships.sql
|
|
2022-01-07 16:46:36.819694 (MainThread): Parsing macros/schema_tests/not_null.sql
|
|
2022-01-07 16:46:36.821990 (MainThread): Parsing macros/schema_tests/unique.sql
|
|
2022-01-07 16:46:36.824326 (MainThread): Parsing macros/schema_tests/accepted_values.sql
|
|
2022-01-07 16:46:36.833664 (MainThread): Partial parsing not enabled
|
|
2022-01-07 16:46:36.881383 (MainThread): Acquiring new snowflake connection "model.my_new_project.stg_model_3".
|
|
2022-01-07 16:46:36.906620 (MainThread): Acquiring new snowflake connection "model.my_new_project.stg_model_2".
|
|
2022-01-07 16:46:36.922673 (MainThread): Acquiring new snowflake connection "model.my_new_project.stg_model_1".
|
|
2022-01-07 16:46:36.935224 (MainThread): Acquiring new snowflake connection "model.my_new_project.fct_model_6".
|
|
2022-01-07 16:46:36.947340 (MainThread): Acquiring new snowflake connection "model.my_new_project.dim_model_7".
|
|
2022-01-07 16:46:36.958962 (MainThread): Acquiring new snowflake connection "model.my_new_project.int_model_4".
|
|
2022-01-07 16:46:36.974065 (MainThread): Acquiring new snowflake connection "model.my_new_project.int_model_5".
|
|
2022-01-07 16:46:37.093952 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'end', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x107e31be0>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x107e3afa0>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x107e3af70>]}
|
|
2022-01-07 16:46:37.094224 (MainThread): Flushing usage events
|
|
2022-01-07 16:46:37.483967 (MainThread): Connection 'model.my_new_project.int_model_5' was properly closed.
|
|
2022-01-07 16:46:37.484263 (MainThread): Encountered an error:
|
|
2022-01-07 16:46:37.484507 (MainThread): Compilation Error in model stg_model_2 (models/staging/stg_model_2.sql)
|
|
Model 'model.my_new_project.stg_model_2' (models/staging/stg_model_2.sql) depends on a source named 'source_2.table_1' which was not found
|
|
2022-01-07 16:46:37.489288 (MainThread): Traceback (most recent call last):
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/main.py", line 124, in main
|
|
results, succeeded = handle_and_check(args)
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/main.py", line 202, in handle_and_check
|
|
task, res = run_from_args(parsed)
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/main.py", line 255, in run_from_args
|
|
results = task.run()
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/task/generate.py", line 213, in run
|
|
compile_results = CompileTask.run(self)
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/task/runnable.py", line 399, in run
|
|
self._runtime_initialize()
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/task/runnable.py", line 118, in _runtime_initialize
|
|
super()._runtime_initialize()
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/task/runnable.py", line 78, in _runtime_initialize
|
|
self.load_manifest()
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/task/runnable.py", line 65, in load_manifest
|
|
self.manifest = get_full_manifest(self.config)
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/perf_utils.py", line 28, in get_full_manifest
|
|
return load_manifest(
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/parser/manifest.py", line 748, in load_manifest
|
|
return ManifestLoader.load_all(config, macro_manifest, macro_hook)
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/parser/manifest.py", line 351, in load_all
|
|
manifest = loader.create_manifest()
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/parser/manifest.py", line 327, in create_manifest
|
|
self.process_manifest(manifest)
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/parser/manifest.py", line 297, in process_manifest
|
|
process_sources(manifest, project_name)
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/parser/manifest.py", line 706, in process_sources
|
|
_process_sources_for_node(manifest, current_project, node)
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/parser/manifest.py", line 689, in _process_sources_for_node
|
|
invalid_source_fail_unless_test(
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/parser/manifest.py", line 406, in invalid_source_fail_unless_test
|
|
source_target_not_found(
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/exceptions.py", line 569, in source_target_not_found
|
|
raise_compiler_error(msg, model)
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/exceptions.py", line 407, in raise_compiler_error
|
|
raise CompilationException(msg, node)
|
|
dbt.exceptions.CompilationException: Compilation Error in model stg_model_2 (models/staging/stg_model_2.sql)
|
|
Model 'model.my_new_project.stg_model_2' (models/staging/stg_model_2.sql) depends on a source named 'source_2.table_1' which was not found
|
|
|
|
2022-01-07 16:46:49.952599 (MainThread): Running with dbt=0.18.1
|
|
2022-01-07 16:46:50.125096 (MainThread): Loading KWallet
|
|
2022-01-07 16:46:50.126735 (MainThread): Loading SecretService
|
|
2022-01-07 16:46:50.127868 (MainThread): Loading Windows
|
|
2022-01-07 16:46:50.129030 (MainThread): Loading chainer
|
|
2022-01-07 16:46:50.129996 (MainThread): Loading macOS
|
|
2022-01-07 16:46:50.549168 (MainThread): running dbt with arguments Namespace(cls=<class 'dbt.task.generate.GenerateTask'>, compile=True, debug=False, defer=None, exclude=None, log_cache_events=False, log_format='default', models=None, partial_parse=None, profile=None, profiles_dir='/Users/daveconnors/.dbt', project_dir=None, record_timing_info=None, rpc_method='docs.generate', selector_name=None, single_threaded=False, state=None, strict=False, target=None, test_new_parser=False, threads=None, use_cache=True, use_colors=None, vars='{}', version_check=True, warn_error=False, which='generate', write_json=True)
|
|
2022-01-07 16:46:50.551719 (MainThread): Tracking: tracking
|
|
2022-01-07 16:46:50.552477 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'start', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10a8721c0>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10a928700>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10a928880>]}
|
|
2022-01-07 16:46:50.583641 (MainThread): Partial parsing not enabled
|
|
2022-01-07 16:46:50.585020 (MainThread): Parsing macros/catalog.sql
|
|
2022-01-07 16:46:50.588243 (MainThread): Parsing macros/adapters.sql
|
|
2022-01-07 16:46:50.626471 (MainThread): Parsing macros/materializations/merge.sql
|
|
2022-01-07 16:46:50.628870 (MainThread): Parsing macros/materializations/view.sql
|
|
2022-01-07 16:46:50.630631 (MainThread): Parsing macros/materializations/table.sql
|
|
2022-01-07 16:46:50.635924 (MainThread): Parsing macros/materializations/incremental.sql
|
|
2022-01-07 16:46:50.651021 (MainThread): Parsing macros/core.sql
|
|
2022-01-07 16:46:50.656362 (MainThread): Parsing macros/materializations/helpers.sql
|
|
2022-01-07 16:46:50.668118 (MainThread): Parsing macros/materializations/snapshot/snapshot_merge.sql
|
|
2022-01-07 16:46:50.670541 (MainThread): Parsing macros/materializations/snapshot/strategies.sql
|
|
2022-01-07 16:46:50.692885 (MainThread): Parsing macros/materializations/snapshot/snapshot.sql
|
|
2022-01-07 16:46:50.727078 (MainThread): Parsing macros/materializations/seed/seed.sql
|
|
2022-01-07 16:46:50.753803 (MainThread): Parsing macros/materializations/incremental/helpers.sql
|
|
2022-01-07 16:46:50.756193 (MainThread): Parsing macros/materializations/incremental/incremental.sql
|
|
2022-01-07 16:46:50.765096 (MainThread): Parsing macros/materializations/common/merge.sql
|
|
2022-01-07 16:46:50.783750 (MainThread): Parsing macros/materializations/table/table.sql
|
|
2022-01-07 16:46:50.792496 (MainThread): Parsing macros/materializations/view/view.sql
|
|
2022-01-07 16:46:50.800458 (MainThread): Parsing macros/materializations/view/create_or_replace_view.sql
|
|
2022-01-07 16:46:50.806909 (MainThread): Parsing macros/etc/get_custom_alias.sql
|
|
2022-01-07 16:46:50.808173 (MainThread): Parsing macros/etc/query.sql
|
|
2022-01-07 16:46:50.809412 (MainThread): Parsing macros/etc/is_incremental.sql
|
|
2022-01-07 16:46:50.811431 (MainThread): Parsing macros/etc/datetime.sql
|
|
2022-01-07 16:46:50.826515 (MainThread): Parsing macros/etc/get_custom_schema.sql
|
|
2022-01-07 16:46:50.829423 (MainThread): Parsing macros/etc/get_custom_database.sql
|
|
2022-01-07 16:46:50.831658 (MainThread): Parsing macros/adapters/common.sql
|
|
2022-01-07 16:46:50.896236 (MainThread): Parsing macros/schema_tests/relationships.sql
|
|
2022-01-07 16:46:50.899237 (MainThread): Parsing macros/schema_tests/not_null.sql
|
|
2022-01-07 16:46:50.901371 (MainThread): Parsing macros/schema_tests/unique.sql
|
|
2022-01-07 16:46:50.903905 (MainThread): Parsing macros/schema_tests/accepted_values.sql
|
|
2022-01-07 16:46:50.914503 (MainThread): Partial parsing not enabled
|
|
2022-01-07 16:46:50.964389 (MainThread): Acquiring new snowflake connection "model.my_new_project.stg_model_3".
|
|
2022-01-07 16:46:50.992884 (MainThread): Acquiring new snowflake connection "model.my_new_project.stg_model_2".
|
|
2022-01-07 16:46:51.008525 (MainThread): Acquiring new snowflake connection "model.my_new_project.stg_model_1".
|
|
2022-01-07 16:46:51.022209 (MainThread): Acquiring new snowflake connection "model.my_new_project.fct_model_6".
|
|
2022-01-07 16:46:51.033566 (MainThread): Acquiring new snowflake connection "model.my_new_project.dim_model_7".
|
|
2022-01-07 16:46:51.045362 (MainThread): Acquiring new snowflake connection "model.my_new_project.int_model_4".
|
|
2022-01-07 16:46:51.059761 (MainThread): Acquiring new snowflake connection "model.my_new_project.int_model_5".
|
|
2022-01-07 16:46:51.215281 (MainThread): [WARNING]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources.
|
|
There are 1 unused configuration paths:
|
|
- models.my_new_project.example
|
|
|
|
2022-01-07 16:46:51.278727 (MainThread): Found 7 models, 0 tests, 0 snapshots, 0 analyses, 143 macros, 0 operations, 0 seed files, 3 sources
|
|
2022-01-07 16:46:51.280914 (MainThread):
|
|
2022-01-07 16:46:51.281568 (MainThread): Acquiring new snowflake connection "master".
|
|
2022-01-07 16:46:51.294195 (ThreadPoolExecutor-0_0): Acquiring new snowflake connection "list_ANALYTICS_ANALYTICS".
|
|
2022-01-07 16:46:51.311939 (ThreadPoolExecutor-0_0): Using snowflake connection "list_ANALYTICS_ANALYTICS".
|
|
2022-01-07 16:46:51.312115 (ThreadPoolExecutor-0_0): On list_ANALYTICS_ANALYTICS: /* {"app": "dbt", "dbt_version": "0.18.1", "profile_name": "default", "target_name": "dev", "connection_name": "list_ANALYTICS_ANALYTICS"} */
|
|
|
|
show terse objects in ANALYTICS.ANALYTICS
|
|
2022-01-07 16:46:51.312213 (ThreadPoolExecutor-0_0): Opening a new connection, currently in state init
|
|
2022-01-07 16:46:52.311128 (ThreadPoolExecutor-0_0): Got an error when attempting to open a snowflake connection: '250001 (08001): Failed to connect to DB: mk43712.us-east-1.snowflakecomputing.com:443. User access disabled. Contact your local system administrator.'
|
|
2022-01-07 16:46:52.311304 (ThreadPoolExecutor-0_0): Error running SQL: /* {"app": "dbt", "dbt_version": "0.18.1", "profile_name": "default", "target_name": "dev", "connection_name": "list_ANALYTICS_ANALYTICS"} */
|
|
|
|
show terse objects in ANALYTICS.ANALYTICS
|
|
2022-01-07 16:46:52.311384 (ThreadPoolExecutor-0_0): Rolling back transaction.
|
|
2022-01-07 16:46:52.311593 (ThreadPoolExecutor-0_0): Error running SQL: macro list_relations_without_caching
|
|
2022-01-07 16:46:52.311701 (ThreadPoolExecutor-0_0): Rolling back transaction.
|
|
2022-01-07 16:46:52.311849 (ThreadPoolExecutor-0_0): On list_ANALYTICS_ANALYTICS: No close available on handle
|
|
2022-01-07 16:46:52.312363 (MainThread): Connection 'master' was properly closed.
|
|
2022-01-07 16:46:52.312462 (MainThread): Connection 'list_ANALYTICS_ANALYTICS' was properly closed.
|
|
2022-01-07 16:46:52.312557 (MainThread): ERROR: Database Error
|
|
250001 (08001): Failed to connect to DB: mk43712.us-east-1.snowflakecomputing.com:443. User access disabled. Contact your local system administrator.
|
|
2022-01-07 16:46:52.312727 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'end', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10b8cabe0>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10b986130>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10bb4baf0>]}
|
|
2022-01-07 16:46:52.312911 (MainThread): Flushing usage events
|
|
2022-01-07 16:47:45.608530 (MainThread): Running with dbt=0.18.1
|
|
2022-01-07 16:47:45.845111 (MainThread): Loading KWallet
|
|
2022-01-07 16:47:45.846211 (MainThread): Loading SecretService
|
|
2022-01-07 16:47:45.846990 (MainThread): Loading Windows
|
|
2022-01-07 16:47:45.847862 (MainThread): Loading chainer
|
|
2022-01-07 16:47:45.848389 (MainThread): Loading macOS
|
|
2022-01-07 16:47:46.367865 (MainThread): running dbt with arguments Namespace(cls=<class 'dbt.task.generate.GenerateTask'>, compile=True, debug=False, defer=None, exclude=None, log_cache_events=False, log_format='default', models=None, partial_parse=None, profile=None, profiles_dir='/Users/daveconnors/.dbt', project_dir=None, record_timing_info=None, rpc_method='docs.generate', selector_name=None, single_threaded=False, state=None, strict=False, target=None, test_new_parser=False, threads=None, use_cache=True, use_colors=None, vars='{}', version_check=True, warn_error=False, which='generate', write_json=True)
|
|
2022-01-07 16:47:46.372057 (MainThread): Tracking: tracking
|
|
2022-01-07 16:47:46.372666 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'start', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x1111b7070>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x11126c7f0>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x11126c790>]}
|
|
2022-01-07 16:47:46.406830 (MainThread): Partial parsing not enabled
|
|
2022-01-07 16:47:46.408627 (MainThread): Parsing macros/catalog.sql
|
|
2022-01-07 16:47:46.411708 (MainThread): Parsing macros/adapters.sql
|
|
2022-01-07 16:47:46.454749 (MainThread): Parsing macros/materializations/merge.sql
|
|
2022-01-07 16:47:46.458089 (MainThread): Parsing macros/materializations/view.sql
|
|
2022-01-07 16:47:46.460597 (MainThread): Parsing macros/materializations/table.sql
|
|
2022-01-07 16:47:46.465891 (MainThread): Parsing macros/materializations/incremental.sql
|
|
2022-01-07 16:47:46.480019 (MainThread): Parsing macros/core.sql
|
|
2022-01-07 16:47:46.491906 (MainThread): Parsing macros/materializations/helpers.sql
|
|
2022-01-07 16:47:46.509164 (MainThread): Parsing macros/materializations/snapshot/snapshot_merge.sql
|
|
2022-01-07 16:47:46.512402 (MainThread): Parsing macros/materializations/snapshot/strategies.sql
|
|
2022-01-07 16:47:46.533942 (MainThread): Parsing macros/materializations/snapshot/snapshot.sql
|
|
2022-01-07 16:47:46.570601 (MainThread): Parsing macros/materializations/seed/seed.sql
|
|
2022-01-07 16:47:46.597179 (MainThread): Parsing macros/materializations/incremental/helpers.sql
|
|
2022-01-07 16:47:46.599825 (MainThread): Parsing macros/materializations/incremental/incremental.sql
|
|
2022-01-07 16:47:46.608085 (MainThread): Parsing macros/materializations/common/merge.sql
|
|
2022-01-07 16:47:46.625694 (MainThread): Parsing macros/materializations/table/table.sql
|
|
2022-01-07 16:47:46.634289 (MainThread): Parsing macros/materializations/view/view.sql
|
|
2022-01-07 16:47:46.642865 (MainThread): Parsing macros/materializations/view/create_or_replace_view.sql
|
|
2022-01-07 16:47:46.648951 (MainThread): Parsing macros/etc/get_custom_alias.sql
|
|
2022-01-07 16:47:46.650499 (MainThread): Parsing macros/etc/query.sql
|
|
2022-01-07 16:47:46.651994 (MainThread): Parsing macros/etc/is_incremental.sql
|
|
2022-01-07 16:47:46.654027 (MainThread): Parsing macros/etc/datetime.sql
|
|
2022-01-07 16:47:46.664964 (MainThread): Parsing macros/etc/get_custom_schema.sql
|
|
2022-01-07 16:47:46.668065 (MainThread): Parsing macros/etc/get_custom_database.sql
|
|
2022-01-07 16:47:46.670472 (MainThread): Parsing macros/adapters/common.sql
|
|
2022-01-07 16:47:46.725704 (MainThread): Parsing macros/schema_tests/relationships.sql
|
|
2022-01-07 16:47:46.728593 (MainThread): Parsing macros/schema_tests/not_null.sql
|
|
2022-01-07 16:47:46.730806 (MainThread): Parsing macros/schema_tests/unique.sql
|
|
2022-01-07 16:47:46.733107 (MainThread): Parsing macros/schema_tests/accepted_values.sql
|
|
2022-01-07 16:47:46.743674 (MainThread): Partial parsing not enabled
|
|
2022-01-07 16:47:46.793426 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_3".
|
|
2022-01-07 16:47:46.818913 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_2".
|
|
2022-01-07 16:47:46.831551 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_1".
|
|
2022-01-07 16:47:46.843535 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.fct_model_6".
|
|
2022-01-07 16:47:46.853613 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.dim_model_7".
|
|
2022-01-07 16:47:46.862956 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.int_model_4".
|
|
2022-01-07 16:47:46.878087 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.int_model_5".
|
|
2022-01-07 16:47:47.097814 (MainThread): Found 7 models, 0 tests, 0 snapshots, 0 analyses, 143 macros, 0 operations, 0 seed files, 3 sources
|
|
2022-01-07 16:47:47.099748 (MainThread):
|
|
2022-01-07 16:47:47.100474 (MainThread): Acquiring new snowflake connection "master".
|
|
2022-01-07 16:47:47.114015 (ThreadPoolExecutor-0_0): Acquiring new snowflake connection "list_analytics_dbt_dconnors".
|
|
2022-01-07 16:47:47.137549 (ThreadPoolExecutor-0_0): Using snowflake connection "list_analytics_dbt_dconnors".
|
|
2022-01-07 16:47:47.137737 (ThreadPoolExecutor-0_0): On list_analytics_dbt_dconnors: /* {"app": "dbt", "dbt_version": "0.18.1", "profile_name": "dbt-learn", "target_name": "dev", "connection_name": "list_analytics_dbt_dconnors"} */
|
|
|
|
show terse objects in analytics.dbt_dconnors
|
|
2022-01-07 16:47:47.137859 (ThreadPoolExecutor-0_0): Opening a new connection, currently in state init
|
|
2022-01-07 16:47:48.310124 (ThreadPoolExecutor-0_0): SQL status: SUCCESS 23 in 1.17 seconds
|
|
2022-01-07 16:47:48.358669 (ThreadPoolExecutor-0_0): On list_analytics_dbt_dconnors: Close
|
|
2022-01-07 16:47:48.571300 (MainThread): 10:47:48 | Concurrency: 1 threads (target='dev')
|
|
2022-01-07 16:47:48.571697 (MainThread): 10:47:48 |
|
|
2022-01-07 16:47:48.575077 (Thread-1): Began running node model.pro_serv_dag_auditing.stg_model_1
|
|
2022-01-07 16:47:48.575543 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_1".
|
|
2022-01-07 16:47:48.575743 (Thread-1): Compiling model.pro_serv_dag_auditing.stg_model_1
|
|
2022-01-07 16:47:48.611868 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.stg_model_1"
|
|
2022-01-07 16:47:48.612749 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:47:48.613089 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:47:48.613534 (Thread-1): Finished running node model.pro_serv_dag_auditing.stg_model_1
|
|
2022-01-07 16:47:48.613681 (Thread-1): Began running node model.pro_serv_dag_auditing.stg_model_2
|
|
2022-01-07 16:47:48.613978 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_2".
|
|
2022-01-07 16:47:48.614426 (Thread-1): Compiling model.pro_serv_dag_auditing.stg_model_2
|
|
2022-01-07 16:47:48.634683 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.stg_model_2"
|
|
2022-01-07 16:47:48.636346 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:47:48.636935 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:47:48.638083 (Thread-1): Finished running node model.pro_serv_dag_auditing.stg_model_2
|
|
2022-01-07 16:47:48.638426 (Thread-1): Began running node model.pro_serv_dag_auditing.stg_model_3
|
|
2022-01-07 16:47:48.639044 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_3".
|
|
2022-01-07 16:47:48.639398 (Thread-1): Compiling model.pro_serv_dag_auditing.stg_model_3
|
|
2022-01-07 16:47:48.657917 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.stg_model_3"
|
|
2022-01-07 16:47:48.658686 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:47:48.659182 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:47:48.660133 (Thread-1): Finished running node model.pro_serv_dag_auditing.stg_model_3
|
|
2022-01-07 16:47:48.660450 (Thread-1): Began running node model.pro_serv_dag_auditing.int_model_4
|
|
2022-01-07 16:47:48.661048 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.int_model_4".
|
|
2022-01-07 16:47:48.661296 (Thread-1): Compiling model.pro_serv_dag_auditing.int_model_4
|
|
2022-01-07 16:47:48.677042 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.int_model_4"
|
|
2022-01-07 16:47:48.678326 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:47:48.678757 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:47:48.679738 (Thread-1): Finished running node model.pro_serv_dag_auditing.int_model_4
|
|
2022-01-07 16:47:48.682172 (Thread-1): Began running node model.pro_serv_dag_auditing.int_model_5
|
|
2022-01-07 16:47:48.683304 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.int_model_5".
|
|
2022-01-07 16:47:48.683898 (Thread-1): Compiling model.pro_serv_dag_auditing.int_model_5
|
|
2022-01-07 16:47:48.697230 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.int_model_5"
|
|
2022-01-07 16:47:48.698112 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:47:48.698659 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:47:48.700894 (Thread-1): Finished running node model.pro_serv_dag_auditing.int_model_5
|
|
2022-01-07 16:47:48.702165 (MainThread): Connection 'master' was properly closed.
|
|
2022-01-07 16:47:48.702307 (MainThread): Connection 'model.pro_serv_dag_auditing.int_model_5' was properly closed.
|
|
2022-01-07 16:47:48.726633 (MainThread): 10:47:48 | Done.
|
|
2022-01-07 16:47:48.737817 (MainThread): Acquiring new snowflake connection "generate_catalog".
|
|
2022-01-07 16:47:48.738256 (MainThread): 10:47:48 | Building catalog
|
|
2022-01-07 16:47:48.826042 (ThreadPoolExecutor-1_0): Acquiring new snowflake connection "analytics.information_schema".
|
|
2022-01-07 16:47:48.836248 (ThreadPoolExecutor-1_0): Using snowflake connection "analytics.information_schema".
|
|
2022-01-07 16:47:48.836418 (ThreadPoolExecutor-1_0): On analytics.information_schema: /* {"app": "dbt", "dbt_version": "0.18.1", "profile_name": "dbt-learn", "target_name": "dev", "connection_name": "analytics.information_schema"} */
|
|
|
|
|
|
with tables as (
|
|
|
|
select
|
|
table_catalog as "table_database",
|
|
table_schema as "table_schema",
|
|
table_name as "table_name",
|
|
table_type as "table_type",
|
|
comment as "table_comment",
|
|
|
|
-- note: this is the _role_ that owns the table
|
|
table_owner as "table_owner",
|
|
|
|
'Clustering Key' as "stats:clustering_key:label",
|
|
clustering_key as "stats:clustering_key:value",
|
|
'The key used to cluster this table' as "stats:clustering_key:description",
|
|
(clustering_key is not null) as "stats:clustering_key:include",
|
|
|
|
'Row Count' as "stats:row_count:label",
|
|
row_count as "stats:row_count:value",
|
|
'An approximate count of rows in this table' as "stats:row_count:description",
|
|
(row_count is not null) as "stats:row_count:include",
|
|
|
|
'Approximate Size' as "stats:bytes:label",
|
|
bytes as "stats:bytes:value",
|
|
'Approximate size of the table as reported by Snowflake' as "stats:bytes:description",
|
|
(bytes is not null) as "stats:bytes:include",
|
|
|
|
'Last Modified' as "stats:last_modified:label",
|
|
to_varchar(convert_timezone('UTC', last_altered), 'yyyy-mm-dd HH24:MI'||'UTC') as "stats:last_modified:value",
|
|
'The timestamp for last update/change' as "stats:last_modified:description",
|
|
(last_altered is not null and table_type='BASE TABLE') as "stats:last_modified:include"
|
|
|
|
from analytics.INFORMATION_SCHEMA.tables
|
|
|
|
),
|
|
|
|
columns as (
|
|
|
|
select
|
|
table_catalog as "table_database",
|
|
table_schema as "table_schema",
|
|
table_name as "table_name",
|
|
|
|
column_name as "column_name",
|
|
ordinal_position as "column_index",
|
|
data_type as "column_type",
|
|
comment as "column_comment"
|
|
|
|
from analytics.INFORMATION_SCHEMA.columns
|
|
)
|
|
|
|
select *
|
|
from tables
|
|
join columns using ("table_database", "table_schema", "table_name")
|
|
where (upper("table_schema") = upper('dbt_dconnors'))
|
|
order by "column_index"
|
|
2022-01-07 16:47:48.836564 (ThreadPoolExecutor-1_0): Opening a new connection, currently in state init
|
|
2022-01-07 16:47:53.089016 (ThreadPoolExecutor-1_0): SQL status: SUCCESS 228 in 4.25 seconds
|
|
2022-01-07 16:47:53.164358 (ThreadPoolExecutor-1_0): On analytics.information_schema: Close
|
|
2022-01-07 16:47:53.362651 (ThreadPoolExecutor-1_0): Acquiring new snowflake connection "real_database.information_schema".
|
|
2022-01-07 16:47:53.372073 (ThreadPoolExecutor-1_0): Using snowflake connection "real_database.information_schema".
|
|
2022-01-07 16:47:53.372665 (ThreadPoolExecutor-1_0): On real_database.information_schema: /* {"app": "dbt", "dbt_version": "0.18.1", "profile_name": "dbt-learn", "target_name": "dev", "connection_name": "real_database.information_schema"} */
|
|
|
|
|
|
with tables as (
|
|
|
|
select
|
|
table_catalog as "table_database",
|
|
table_schema as "table_schema",
|
|
table_name as "table_name",
|
|
table_type as "table_type",
|
|
comment as "table_comment",
|
|
|
|
-- note: this is the _role_ that owns the table
|
|
table_owner as "table_owner",
|
|
|
|
'Clustering Key' as "stats:clustering_key:label",
|
|
clustering_key as "stats:clustering_key:value",
|
|
'The key used to cluster this table' as "stats:clustering_key:description",
|
|
(clustering_key is not null) as "stats:clustering_key:include",
|
|
|
|
'Row Count' as "stats:row_count:label",
|
|
row_count as "stats:row_count:value",
|
|
'An approximate count of rows in this table' as "stats:row_count:description",
|
|
(row_count is not null) as "stats:row_count:include",
|
|
|
|
'Approximate Size' as "stats:bytes:label",
|
|
bytes as "stats:bytes:value",
|
|
'Approximate size of the table as reported by Snowflake' as "stats:bytes:description",
|
|
(bytes is not null) as "stats:bytes:include",
|
|
|
|
'Last Modified' as "stats:last_modified:label",
|
|
to_varchar(convert_timezone('UTC', last_altered), 'yyyy-mm-dd HH24:MI'||'UTC') as "stats:last_modified:value",
|
|
'The timestamp for last update/change' as "stats:last_modified:description",
|
|
(last_altered is not null and table_type='BASE TABLE') as "stats:last_modified:include"
|
|
|
|
from real_database.INFORMATION_SCHEMA.tables
|
|
|
|
),
|
|
|
|
columns as (
|
|
|
|
select
|
|
table_catalog as "table_database",
|
|
table_schema as "table_schema",
|
|
table_name as "table_name",
|
|
|
|
column_name as "column_name",
|
|
ordinal_position as "column_index",
|
|
data_type as "column_type",
|
|
comment as "column_comment"
|
|
|
|
from real_database.INFORMATION_SCHEMA.columns
|
|
)
|
|
|
|
select *
|
|
from tables
|
|
join columns using ("table_database", "table_schema", "table_name")
|
|
where (upper("table_schema") = upper('real_schema'))
|
|
order by "column_index"
|
|
2022-01-07 16:47:53.377416 (ThreadPoolExecutor-1_0): Opening a new connection, currently in state closed
|
|
2022-01-07 16:47:54.377240 (ThreadPoolExecutor-1_0): Snowflake query id: 01a1794f-0600-fe34-0007-97830178d1ba
|
|
2022-01-07 16:47:54.377511 (ThreadPoolExecutor-1_0): Snowflake error: 002003 (02000): SQL compilation error:
|
|
Database 'REAL_DATABASE' does not exist or not authorized.
|
|
2022-01-07 16:47:54.377688 (ThreadPoolExecutor-1_0): Error running SQL: macro get_catalog
|
|
2022-01-07 16:47:54.377777 (ThreadPoolExecutor-1_0): Rolling back transaction.
|
|
2022-01-07 16:47:54.377918 (ThreadPoolExecutor-1_0): On real_database.information_schema: Close
|
|
2022-01-07 16:47:54.490966 (MainThread): Encountered an error while generating catalog: Database Error
|
|
002003 (02000): SQL compilation error:
|
|
Database 'REAL_DATABASE' does not exist or not authorized.
|
|
2022-01-07 16:47:54.692021 (MainThread): dbt encountered 1 failure while writing the catalog
|
|
2022-01-07 16:47:54.692432 (MainThread): 10:47:54 | Catalog written to /Users/daveconnors/dev/proserv/pro-serv-dag-auditing/target/catalog.json
|
|
2022-01-07 16:47:54.693296 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'end', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x1111b7070>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x112019f40>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x11211e250>]}
|
|
2022-01-07 16:47:54.693823 (MainThread): Flushing usage events
|
|
2022-01-07 16:47:55.102492 (MainThread): Connection 'generate_catalog' was properly closed.
|
|
2022-01-07 16:47:55.103070 (MainThread): Connection 'real_database.information_schema' was properly closed.
|
|
2022-01-07 16:48:45.824696 (MainThread): Running with dbt=0.18.1
|
|
2022-01-07 16:48:46.115661 (MainThread): Loading KWallet
|
|
2022-01-07 16:48:46.117169 (MainThread): Loading SecretService
|
|
2022-01-07 16:48:46.118344 (MainThread): Loading Windows
|
|
2022-01-07 16:48:46.119659 (MainThread): Loading chainer
|
|
2022-01-07 16:48:46.120577 (MainThread): Loading macOS
|
|
2022-01-07 16:48:46.689477 (MainThread): running dbt with arguments Namespace(cls=<class 'dbt.task.serve.ServeTask'>, debug=False, defer=None, log_cache_events=False, log_format='default', open_browser=True, partial_parse=None, port=8080, profile=None, profiles_dir='/Users/daveconnors/.dbt', project_dir=None, record_timing_info=None, rpc_method=None, single_threaded=False, state=None, strict=False, target=None, test_new_parser=False, use_cache=True, use_colors=None, vars='{}', warn_error=False, which='serve', write_json=True)
|
|
2022-01-07 16:48:46.692949 (MainThread): Tracking: tracking
|
|
2022-01-07 16:48:46.693766 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'start', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10b2ec7c0>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10b2ec790>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10b414d90>]}
|
|
2022-01-07 16:48:46.697632 (MainThread): Serving docs at 0.0.0.0:8080
|
|
2022-01-07 16:48:46.697886 (MainThread): To access from your browser, navigate to: http://localhost:8080
|
|
2022-01-07 16:48:46.698025 (MainThread): Press Ctrl+C to exit.
|
|
|
|
|
|
2022-01-07 16:50:25.192899 (MainThread): Running with dbt=0.18.1
|
|
2022-01-07 16:50:25.456715 (MainThread): Loading KWallet
|
|
2022-01-07 16:50:25.457821 (MainThread): Loading SecretService
|
|
2022-01-07 16:50:25.458593 (MainThread): Loading Windows
|
|
2022-01-07 16:50:25.459745 (MainThread): Loading chainer
|
|
2022-01-07 16:50:25.460578 (MainThread): Loading macOS
|
|
2022-01-07 16:50:26.192535 (MainThread): running dbt with arguments Namespace(cls=<class 'dbt.task.serve.ServeTask'>, debug=False, defer=None, log_cache_events=False, log_format='default', open_browser=True, partial_parse=None, port=8080, profile=None, profiles_dir='/Users/daveconnors/.dbt', project_dir=None, record_timing_info=None, rpc_method=None, single_threaded=False, state=None, strict=False, target=None, test_new_parser=False, use_cache=True, use_colors=None, vars='{}', warn_error=False, which='serve', write_json=True)
|
|
2022-01-07 16:50:26.208248 (MainThread): Tracking: tracking
|
|
2022-01-07 16:50:26.211536 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'start', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x110696d30>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x110696d90>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x1106bcc10>]}
|
|
2022-01-07 16:50:26.221970 (MainThread): Serving docs at 0.0.0.0:8080
|
|
2022-01-07 16:50:26.222772 (MainThread): To access from your browser, navigate to: http://localhost:8080
|
|
2022-01-07 16:50:26.223338 (MainThread): Press Ctrl+C to exit.
|
|
|
|
|
|
2022-01-07 16:50:26.224254 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'end', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x1106bcaf0>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x1106bc100>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x111189d30>]}
|
|
2022-01-07 16:50:26.225184 (MainThread): Flushing usage events
|
|
2022-01-07 16:50:26.931972 (MainThread): Encountered an error:
|
|
2022-01-07 16:50:26.932228 (MainThread): [Errno 48] Address already in use
|
|
2022-01-07 16:50:26.935834 (MainThread): Traceback (most recent call last):
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/main.py", line 124, in main
|
|
results, succeeded = handle_and_check(args)
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/main.py", line 202, in handle_and_check
|
|
task, res = run_from_args(parsed)
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/main.py", line 255, in run_from_args
|
|
results = task.run()
|
|
File "/usr/local/Cellar/dbt/0.18.1_1/libexec/lib/python3.8/site-packages/dbt/task/serve.py", line 29, in run
|
|
httpd = TCPServer( # type: ignore
|
|
File "/usr/local/Cellar/python@3.8/3.8.8_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/socketserver.py", line 452, in __init__
|
|
self.server_bind()
|
|
File "/usr/local/Cellar/python@3.8/3.8.8_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/socketserver.py", line 466, in server_bind
|
|
self.socket.bind(self.server_address)
|
|
OSError: [Errno 48] Address already in use
|
|
|
|
2022-01-07 16:50:32.210145 (MainThread): Flushing usage events
|
|
2022-01-07 16:50:32.722152 (MainThread): ctrl-c
|
|
2022-01-07 16:50:42.699292 (MainThread): Running with dbt=0.18.1
|
|
2022-01-07 16:50:42.905584 (MainThread): Loading KWallet
|
|
2022-01-07 16:50:42.906662 (MainThread): Loading SecretService
|
|
2022-01-07 16:50:42.907574 (MainThread): Loading Windows
|
|
2022-01-07 16:50:42.908879 (MainThread): Loading chainer
|
|
2022-01-07 16:50:42.909894 (MainThread): Loading macOS
|
|
2022-01-07 16:50:43.407048 (MainThread): running dbt with arguments Namespace(cls=<class 'dbt.task.serve.ServeTask'>, debug=False, defer=None, log_cache_events=False, log_format='default', open_browser=True, partial_parse=None, port=8080, profile=None, profiles_dir='/Users/daveconnors/.dbt', project_dir=None, record_timing_info=None, rpc_method=None, single_threaded=False, state=None, strict=False, target=None, test_new_parser=False, use_cache=True, use_colors=None, vars='{}', warn_error=False, which='serve', write_json=True)
|
|
2022-01-07 16:50:43.411082 (MainThread): Tracking: tracking
|
|
2022-01-07 16:50:43.411976 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'start', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10b911790>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10b911760>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10ba3ad60>]}
|
|
2022-01-07 16:50:43.417011 (MainThread): Serving docs at 0.0.0.0:8080
|
|
2022-01-07 16:50:43.417353 (MainThread): To access from your browser, navigate to: http://localhost:8080
|
|
2022-01-07 16:50:43.417574 (MainThread): Press Ctrl+C to exit.
|
|
|
|
|
|
2022-01-07 16:50:59.595573 (MainThread): Running with dbt=0.18.1
|
|
2022-01-07 16:50:59.794463 (MainThread): Loading KWallet
|
|
2022-01-07 16:50:59.796424 (MainThread): Loading SecretService
|
|
2022-01-07 16:50:59.798076 (MainThread): Loading Windows
|
|
2022-01-07 16:50:59.799571 (MainThread): Loading chainer
|
|
2022-01-07 16:50:59.800338 (MainThread): Loading macOS
|
|
2022-01-07 16:51:00.321332 (MainThread): running dbt with arguments Namespace(cls=<class 'dbt.task.generate.GenerateTask'>, compile=True, debug=False, defer=None, exclude=None, log_cache_events=False, log_format='default', models=None, partial_parse=None, profile=None, profiles_dir='/Users/daveconnors/.dbt', project_dir=None, record_timing_info=None, rpc_method='docs.generate', selector_name=None, single_threaded=False, state=None, strict=False, target=None, test_new_parser=False, threads=None, use_cache=True, use_colors=None, vars='{}', version_check=True, warn_error=False, which='generate', write_json=True)
|
|
2022-01-07 16:51:00.324178 (MainThread): Tracking: tracking
|
|
2022-01-07 16:51:00.324834 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'start', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x1113a83a0>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x11145d700>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x11145d880>]}
|
|
2022-01-07 16:51:00.358920 (MainThread): Partial parsing not enabled
|
|
2022-01-07 16:51:00.361380 (MainThread): Parsing macros/catalog.sql
|
|
2022-01-07 16:51:00.385734 (MainThread): Parsing macros/adapters.sql
|
|
2022-01-07 16:51:00.427337 (MainThread): Parsing macros/materializations/merge.sql
|
|
2022-01-07 16:51:00.430316 (MainThread): Parsing macros/materializations/view.sql
|
|
2022-01-07 16:51:00.432967 (MainThread): Parsing macros/materializations/table.sql
|
|
2022-01-07 16:51:00.438408 (MainThread): Parsing macros/materializations/incremental.sql
|
|
2022-01-07 16:51:00.453888 (MainThread): Parsing macros/core.sql
|
|
2022-01-07 16:51:00.460075 (MainThread): Parsing macros/materializations/helpers.sql
|
|
2022-01-07 16:51:00.471499 (MainThread): Parsing macros/materializations/snapshot/snapshot_merge.sql
|
|
2022-01-07 16:51:00.474709 (MainThread): Parsing macros/materializations/snapshot/strategies.sql
|
|
2022-01-07 16:51:00.500970 (MainThread): Parsing macros/materializations/snapshot/snapshot.sql
|
|
2022-01-07 16:51:00.541427 (MainThread): Parsing macros/materializations/seed/seed.sql
|
|
2022-01-07 16:51:00.569702 (MainThread): Parsing macros/materializations/incremental/helpers.sql
|
|
2022-01-07 16:51:00.572194 (MainThread): Parsing macros/materializations/incremental/incremental.sql
|
|
2022-01-07 16:51:00.580085 (MainThread): Parsing macros/materializations/common/merge.sql
|
|
2022-01-07 16:51:00.598373 (MainThread): Parsing macros/materializations/table/table.sql
|
|
2022-01-07 16:51:00.607270 (MainThread): Parsing macros/materializations/view/view.sql
|
|
2022-01-07 16:51:00.615251 (MainThread): Parsing macros/materializations/view/create_or_replace_view.sql
|
|
2022-01-07 16:51:00.622038 (MainThread): Parsing macros/etc/get_custom_alias.sql
|
|
2022-01-07 16:51:00.623742 (MainThread): Parsing macros/etc/query.sql
|
|
2022-01-07 16:51:00.626112 (MainThread): Parsing macros/etc/is_incremental.sql
|
|
2022-01-07 16:51:00.629000 (MainThread): Parsing macros/etc/datetime.sql
|
|
2022-01-07 16:51:00.640709 (MainThread): Parsing macros/etc/get_custom_schema.sql
|
|
2022-01-07 16:51:00.644201 (MainThread): Parsing macros/etc/get_custom_database.sql
|
|
2022-01-07 16:51:00.646476 (MainThread): Parsing macros/adapters/common.sql
|
|
2022-01-07 16:51:00.700082 (MainThread): Parsing macros/schema_tests/relationships.sql
|
|
2022-01-07 16:51:00.702621 (MainThread): Parsing macros/schema_tests/not_null.sql
|
|
2022-01-07 16:51:00.704635 (MainThread): Parsing macros/schema_tests/unique.sql
|
|
2022-01-07 16:51:00.707721 (MainThread): Parsing macros/schema_tests/accepted_values.sql
|
|
2022-01-07 16:51:00.717717 (MainThread): Partial parsing not enabled
|
|
2022-01-07 16:51:00.770527 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_3".
|
|
2022-01-07 16:51:00.795844 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_2".
|
|
2022-01-07 16:51:00.810201 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_1".
|
|
2022-01-07 16:51:00.822613 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.fct_model_6".
|
|
2022-01-07 16:51:00.833743 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.dim_model_7".
|
|
2022-01-07 16:51:00.844286 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.int_model_4".
|
|
2022-01-07 16:51:00.857196 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.int_model_5".
|
|
2022-01-07 16:51:01.084066 (MainThread): Found 7 models, 0 tests, 0 snapshots, 0 analyses, 143 macros, 0 operations, 0 seed files, 3 sources
|
|
2022-01-07 16:51:01.085534 (MainThread):
|
|
2022-01-07 16:51:01.085922 (MainThread): Acquiring new snowflake connection "master".
|
|
2022-01-07 16:51:01.097564 (ThreadPoolExecutor-0_0): Acquiring new snowflake connection "list_analytics_dbt_dconnors".
|
|
2022-01-07 16:51:01.117683 (ThreadPoolExecutor-0_0): Using snowflake connection "list_analytics_dbt_dconnors".
|
|
2022-01-07 16:51:01.117835 (ThreadPoolExecutor-0_0): On list_analytics_dbt_dconnors: /* {"app": "dbt", "dbt_version": "0.18.1", "profile_name": "dbt-learn", "target_name": "dev", "connection_name": "list_analytics_dbt_dconnors"} */
|
|
|
|
show terse objects in analytics.dbt_dconnors
|
|
2022-01-07 16:51:01.117927 (ThreadPoolExecutor-0_0): Opening a new connection, currently in state init
|
|
2022-01-07 16:51:02.682875 (ThreadPoolExecutor-0_0): SQL status: SUCCESS 23 in 1.56 seconds
|
|
2022-01-07 16:51:02.717554 (ThreadPoolExecutor-0_0): On list_analytics_dbt_dconnors: Close
|
|
2022-01-07 16:51:02.912329 (MainThread): 10:51:02 | Concurrency: 1 threads (target='dev')
|
|
2022-01-07 16:51:02.912754 (MainThread): 10:51:02 |
|
|
2022-01-07 16:51:02.916769 (Thread-1): Began running node model.pro_serv_dag_auditing.stg_model_1
|
|
2022-01-07 16:51:02.917414 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_1".
|
|
2022-01-07 16:51:02.917685 (Thread-1): Compiling model.pro_serv_dag_auditing.stg_model_1
|
|
2022-01-07 16:51:02.956237 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.stg_model_1"
|
|
2022-01-07 16:51:02.956870 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:51:02.957384 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:51:02.957985 (Thread-1): Finished running node model.pro_serv_dag_auditing.stg_model_1
|
|
2022-01-07 16:51:02.958234 (Thread-1): Began running node model.pro_serv_dag_auditing.stg_model_2
|
|
2022-01-07 16:51:02.959237 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_2".
|
|
2022-01-07 16:51:02.959537 (Thread-1): Compiling model.pro_serv_dag_auditing.stg_model_2
|
|
2022-01-07 16:51:02.976444 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.stg_model_2"
|
|
2022-01-07 16:51:02.977110 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:51:02.977634 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:51:02.978139 (Thread-1): Finished running node model.pro_serv_dag_auditing.stg_model_2
|
|
2022-01-07 16:51:02.978378 (Thread-1): Began running node model.pro_serv_dag_auditing.stg_model_3
|
|
2022-01-07 16:51:02.978705 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_3".
|
|
2022-01-07 16:51:02.978836 (Thread-1): Compiling model.pro_serv_dag_auditing.stg_model_3
|
|
2022-01-07 16:51:02.989000 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.stg_model_3"
|
|
2022-01-07 16:51:02.989576 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:51:02.990135 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:51:02.991236 (Thread-1): Finished running node model.pro_serv_dag_auditing.stg_model_3
|
|
2022-01-07 16:51:02.991575 (Thread-1): Began running node model.pro_serv_dag_auditing.int_model_4
|
|
2022-01-07 16:51:02.991977 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.int_model_4".
|
|
2022-01-07 16:51:02.992263 (Thread-1): Compiling model.pro_serv_dag_auditing.int_model_4
|
|
2022-01-07 16:51:03.008000 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.int_model_4"
|
|
2022-01-07 16:51:03.009737 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:51:03.010278 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:51:03.010945 (Thread-1): Finished running node model.pro_serv_dag_auditing.int_model_4
|
|
2022-01-07 16:51:03.011901 (Thread-1): Began running node model.pro_serv_dag_auditing.int_model_5
|
|
2022-01-07 16:51:03.012392 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.int_model_5".
|
|
2022-01-07 16:51:03.012548 (Thread-1): Compiling model.pro_serv_dag_auditing.int_model_5
|
|
2022-01-07 16:51:03.024181 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.int_model_5"
|
|
2022-01-07 16:51:03.026843 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:51:03.028551 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:51:03.030958 (Thread-1): Finished running node model.pro_serv_dag_auditing.int_model_5
|
|
2022-01-07 16:51:03.033124 (MainThread): Connection 'master' was properly closed.
|
|
2022-01-07 16:51:03.033332 (MainThread): Connection 'model.pro_serv_dag_auditing.int_model_5' was properly closed.
|
|
2022-01-07 16:51:03.058023 (MainThread): 10:51:03 | Done.
|
|
2022-01-07 16:51:03.061615 (MainThread): Acquiring new snowflake connection "generate_catalog".
|
|
2022-01-07 16:51:03.061772 (MainThread): 10:51:03 | Building catalog
|
|
2022-01-07 16:51:03.117722 (ThreadPoolExecutor-1_0): Acquiring new snowflake connection "analytics.information_schema".
|
|
2022-01-07 16:51:03.136491 (ThreadPoolExecutor-1_0): Using snowflake connection "analytics.information_schema".
|
|
2022-01-07 16:51:03.136702 (ThreadPoolExecutor-1_0): On analytics.information_schema: /* {"app": "dbt", "dbt_version": "0.18.1", "profile_name": "dbt-learn", "target_name": "dev", "connection_name": "analytics.information_schema"} */
|
|
|
|
|
|
with tables as (
|
|
|
|
select
|
|
table_catalog as "table_database",
|
|
table_schema as "table_schema",
|
|
table_name as "table_name",
|
|
table_type as "table_type",
|
|
comment as "table_comment",
|
|
|
|
-- note: this is the _role_ that owns the table
|
|
table_owner as "table_owner",
|
|
|
|
'Clustering Key' as "stats:clustering_key:label",
|
|
clustering_key as "stats:clustering_key:value",
|
|
'The key used to cluster this table' as "stats:clustering_key:description",
|
|
(clustering_key is not null) as "stats:clustering_key:include",
|
|
|
|
'Row Count' as "stats:row_count:label",
|
|
row_count as "stats:row_count:value",
|
|
'An approximate count of rows in this table' as "stats:row_count:description",
|
|
(row_count is not null) as "stats:row_count:include",
|
|
|
|
'Approximate Size' as "stats:bytes:label",
|
|
bytes as "stats:bytes:value",
|
|
'Approximate size of the table as reported by Snowflake' as "stats:bytes:description",
|
|
(bytes is not null) as "stats:bytes:include",
|
|
|
|
'Last Modified' as "stats:last_modified:label",
|
|
to_varchar(convert_timezone('UTC', last_altered), 'yyyy-mm-dd HH24:MI'||'UTC') as "stats:last_modified:value",
|
|
'The timestamp for last update/change' as "stats:last_modified:description",
|
|
(last_altered is not null and table_type='BASE TABLE') as "stats:last_modified:include"
|
|
|
|
from analytics.INFORMATION_SCHEMA.tables
|
|
|
|
),
|
|
|
|
columns as (
|
|
|
|
select
|
|
table_catalog as "table_database",
|
|
table_schema as "table_schema",
|
|
table_name as "table_name",
|
|
|
|
column_name as "column_name",
|
|
ordinal_position as "column_index",
|
|
data_type as "column_type",
|
|
comment as "column_comment"
|
|
|
|
from analytics.INFORMATION_SCHEMA.columns
|
|
)
|
|
|
|
select *
|
|
from tables
|
|
join columns using ("table_database", "table_schema", "table_name")
|
|
where (upper("table_schema") = upper('dbt_dconnors'))
|
|
order by "column_index"
|
|
2022-01-07 16:51:03.136873 (ThreadPoolExecutor-1_0): Opening a new connection, currently in state init
|
|
2022-01-07 16:51:07.339599 (ThreadPoolExecutor-1_0): SQL status: SUCCESS 228 in 4.20 seconds
|
|
2022-01-07 16:51:07.415347 (ThreadPoolExecutor-1_0): On analytics.information_schema: Close
|
|
2022-01-07 16:51:07.612732 (ThreadPoolExecutor-1_0): Acquiring new snowflake connection "real_database.information_schema".
|
|
2022-01-07 16:51:07.622577 (ThreadPoolExecutor-1_0): Using snowflake connection "real_database.information_schema".
|
|
2022-01-07 16:51:07.622808 (ThreadPoolExecutor-1_0): On real_database.information_schema: /* {"app": "dbt", "dbt_version": "0.18.1", "profile_name": "dbt-learn", "target_name": "dev", "connection_name": "real_database.information_schema"} */
|
|
|
|
|
|
with tables as (
|
|
|
|
select
|
|
table_catalog as "table_database",
|
|
table_schema as "table_schema",
|
|
table_name as "table_name",
|
|
table_type as "table_type",
|
|
comment as "table_comment",
|
|
|
|
-- note: this is the _role_ that owns the table
|
|
table_owner as "table_owner",
|
|
|
|
'Clustering Key' as "stats:clustering_key:label",
|
|
clustering_key as "stats:clustering_key:value",
|
|
'The key used to cluster this table' as "stats:clustering_key:description",
|
|
(clustering_key is not null) as "stats:clustering_key:include",
|
|
|
|
'Row Count' as "stats:row_count:label",
|
|
row_count as "stats:row_count:value",
|
|
'An approximate count of rows in this table' as "stats:row_count:description",
|
|
(row_count is not null) as "stats:row_count:include",
|
|
|
|
'Approximate Size' as "stats:bytes:label",
|
|
bytes as "stats:bytes:value",
|
|
'Approximate size of the table as reported by Snowflake' as "stats:bytes:description",
|
|
(bytes is not null) as "stats:bytes:include",
|
|
|
|
'Last Modified' as "stats:last_modified:label",
|
|
to_varchar(convert_timezone('UTC', last_altered), 'yyyy-mm-dd HH24:MI'||'UTC') as "stats:last_modified:value",
|
|
'The timestamp for last update/change' as "stats:last_modified:description",
|
|
(last_altered is not null and table_type='BASE TABLE') as "stats:last_modified:include"
|
|
|
|
from real_database.INFORMATION_SCHEMA.tables
|
|
|
|
),
|
|
|
|
columns as (
|
|
|
|
select
|
|
table_catalog as "table_database",
|
|
table_schema as "table_schema",
|
|
table_name as "table_name",
|
|
|
|
column_name as "column_name",
|
|
ordinal_position as "column_index",
|
|
data_type as "column_type",
|
|
comment as "column_comment"
|
|
|
|
from real_database.INFORMATION_SCHEMA.columns
|
|
)
|
|
|
|
select *
|
|
from tables
|
|
join columns using ("table_database", "table_schema", "table_name")
|
|
where (upper("table_schema") = upper('real_schema'))
|
|
order by "column_index"
|
|
2022-01-07 16:51:07.622946 (ThreadPoolExecutor-1_0): Opening a new connection, currently in state closed
|
|
2022-01-07 16:51:08.490929 (ThreadPoolExecutor-1_0): Snowflake query id: 01a17953-0600-fda7-0007-9783017862ee
|
|
2022-01-07 16:51:08.491132 (ThreadPoolExecutor-1_0): Snowflake error: 002003 (02000): SQL compilation error:
|
|
Database 'REAL_DATABASE' does not exist or not authorized.
|
|
2022-01-07 16:51:08.491305 (ThreadPoolExecutor-1_0): Error running SQL: macro get_catalog
|
|
2022-01-07 16:51:08.491400 (ThreadPoolExecutor-1_0): Rolling back transaction.
|
|
2022-01-07 16:51:08.491532 (ThreadPoolExecutor-1_0): On real_database.information_schema: Close
|
|
2022-01-07 16:51:08.618139 (MainThread): Encountered an error while generating catalog: Database Error
|
|
002003 (02000): SQL compilation error:
|
|
Database 'REAL_DATABASE' does not exist or not authorized.
|
|
2022-01-07 16:51:08.791184 (MainThread): dbt encountered 1 failure while writing the catalog
|
|
2022-01-07 16:51:08.791774 (MainThread): 10:51:08 | Catalog written to /Users/daveconnors/dev/proserv/pro-serv-dag-auditing/target/catalog.json
|
|
2022-01-07 16:51:08.792985 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'end', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x1113a83a0>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x112691f70>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x112331af0>]}
|
|
2022-01-07 16:51:08.793591 (MainThread): Flushing usage events
|
|
2022-01-07 16:51:09.093728 (MainThread): Connection 'generate_catalog' was properly closed.
|
|
2022-01-07 16:51:09.093978 (MainThread): Connection 'real_database.information_schema' was properly closed.
|
|
2022-01-07 16:55:17.580655 (MainThread): Running with dbt=0.18.1
|
|
2022-01-07 16:55:17.825619 (MainThread): Loading KWallet
|
|
2022-01-07 16:55:17.826744 (MainThread): Loading SecretService
|
|
2022-01-07 16:55:17.827519 (MainThread): Loading Windows
|
|
2022-01-07 16:55:17.828462 (MainThread): Loading chainer
|
|
2022-01-07 16:55:17.829115 (MainThread): Loading macOS
|
|
2022-01-07 16:55:18.410302 (MainThread): running dbt with arguments Namespace(cls=<class 'dbt.task.generate.GenerateTask'>, compile=True, debug=False, defer=None, exclude=None, log_cache_events=False, log_format='default', models=None, partial_parse=None, profile=None, profiles_dir='/Users/daveconnors/.dbt', project_dir=None, record_timing_info=None, rpc_method='docs.generate', selector_name=None, single_threaded=False, state=None, strict=False, target=None, test_new_parser=False, threads=None, use_cache=True, use_colors=None, vars='{}', version_check=True, warn_error=False, which='generate', write_json=True)
|
|
2022-01-07 16:55:18.412567 (MainThread): Tracking: tracking
|
|
2022-01-07 16:55:18.413054 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'start', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10482d070>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x1048e37f0>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x1048e3790>]}
|
|
2022-01-07 16:55:18.454395 (MainThread): Partial parsing not enabled
|
|
2022-01-07 16:55:18.457539 (MainThread): Parsing macros/catalog.sql
|
|
2022-01-07 16:55:18.461308 (MainThread): Parsing macros/adapters.sql
|
|
2022-01-07 16:55:18.510466 (MainThread): Parsing macros/materializations/merge.sql
|
|
2022-01-07 16:55:18.513256 (MainThread): Parsing macros/materializations/view.sql
|
|
2022-01-07 16:55:18.515556 (MainThread): Parsing macros/materializations/table.sql
|
|
2022-01-07 16:55:18.522203 (MainThread): Parsing macros/materializations/incremental.sql
|
|
2022-01-07 16:55:18.539069 (MainThread): Parsing macros/core.sql
|
|
2022-01-07 16:55:18.543796 (MainThread): Parsing macros/materializations/helpers.sql
|
|
2022-01-07 16:55:18.561352 (MainThread): Parsing macros/materializations/snapshot/snapshot_merge.sql
|
|
2022-01-07 16:55:18.564279 (MainThread): Parsing macros/materializations/snapshot/strategies.sql
|
|
2022-01-07 16:55:18.589857 (MainThread): Parsing macros/materializations/snapshot/snapshot.sql
|
|
2022-01-07 16:55:18.630542 (MainThread): Parsing macros/materializations/seed/seed.sql
|
|
2022-01-07 16:55:18.667461 (MainThread): Parsing macros/materializations/incremental/helpers.sql
|
|
2022-01-07 16:55:18.670719 (MainThread): Parsing macros/materializations/incremental/incremental.sql
|
|
2022-01-07 16:55:18.681528 (MainThread): Parsing macros/materializations/common/merge.sql
|
|
2022-01-07 16:55:18.705893 (MainThread): Parsing macros/materializations/table/table.sql
|
|
2022-01-07 16:55:18.716213 (MainThread): Parsing macros/materializations/view/view.sql
|
|
2022-01-07 16:55:18.727955 (MainThread): Parsing macros/materializations/view/create_or_replace_view.sql
|
|
2022-01-07 16:55:18.738829 (MainThread): Parsing macros/etc/get_custom_alias.sql
|
|
2022-01-07 16:55:18.740570 (MainThread): Parsing macros/etc/query.sql
|
|
2022-01-07 16:55:18.742277 (MainThread): Parsing macros/etc/is_incremental.sql
|
|
2022-01-07 16:55:18.744725 (MainThread): Parsing macros/etc/datetime.sql
|
|
2022-01-07 16:55:18.757229 (MainThread): Parsing macros/etc/get_custom_schema.sql
|
|
2022-01-07 16:55:18.760169 (MainThread): Parsing macros/etc/get_custom_database.sql
|
|
2022-01-07 16:55:18.762581 (MainThread): Parsing macros/adapters/common.sql
|
|
2022-01-07 16:55:18.832860 (MainThread): Parsing macros/schema_tests/relationships.sql
|
|
2022-01-07 16:55:18.835719 (MainThread): Parsing macros/schema_tests/not_null.sql
|
|
2022-01-07 16:55:18.837928 (MainThread): Parsing macros/schema_tests/unique.sql
|
|
2022-01-07 16:55:18.840199 (MainThread): Parsing macros/schema_tests/accepted_values.sql
|
|
2022-01-07 16:55:18.849914 (MainThread): Partial parsing not enabled
|
|
2022-01-07 16:55:18.898483 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_3".
|
|
2022-01-07 16:55:18.924485 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_2".
|
|
2022-01-07 16:55:18.939475 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_1".
|
|
2022-01-07 16:55:18.951534 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.fct_model_6".
|
|
2022-01-07 16:55:18.963126 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.dim_model_7".
|
|
2022-01-07 16:55:18.975948 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.int_model_4".
|
|
2022-01-07 16:55:18.992187 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.int_model_5".
|
|
2022-01-07 16:55:19.216563 (MainThread): Found 7 models, 0 tests, 0 snapshots, 0 analyses, 143 macros, 0 operations, 0 seed files, 3 sources
|
|
2022-01-07 16:55:19.218213 (MainThread):
|
|
2022-01-07 16:55:19.218598 (MainThread): Acquiring new snowflake connection "master".
|
|
2022-01-07 16:55:19.229455 (ThreadPoolExecutor-0_0): Acquiring new snowflake connection "list_analytics_dbt_dconnors".
|
|
2022-01-07 16:55:19.246549 (ThreadPoolExecutor-0_0): Using snowflake connection "list_analytics_dbt_dconnors".
|
|
2022-01-07 16:55:19.246705 (ThreadPoolExecutor-0_0): On list_analytics_dbt_dconnors: /* {"app": "dbt", "dbt_version": "0.18.1", "profile_name": "dbt-learn", "target_name": "dev", "connection_name": "list_analytics_dbt_dconnors"} */
|
|
|
|
show terse objects in analytics.dbt_dconnors
|
|
2022-01-07 16:55:19.246800 (ThreadPoolExecutor-0_0): Opening a new connection, currently in state init
|
|
2022-01-07 16:55:20.436978 (ThreadPoolExecutor-0_0): SQL status: SUCCESS 23 in 1.19 seconds
|
|
2022-01-07 16:55:20.471599 (ThreadPoolExecutor-0_0): On list_analytics_dbt_dconnors: Close
|
|
2022-01-07 16:55:20.671216 (MainThread): 10:55:20 | Concurrency: 1 threads (target='dev')
|
|
2022-01-07 16:55:20.671677 (MainThread): 10:55:20 |
|
|
2022-01-07 16:55:20.674722 (Thread-1): Began running node model.pro_serv_dag_auditing.stg_model_1
|
|
2022-01-07 16:55:20.675322 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_1".
|
|
2022-01-07 16:55:20.675497 (Thread-1): Compiling model.pro_serv_dag_auditing.stg_model_1
|
|
2022-01-07 16:55:20.710285 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.stg_model_1"
|
|
2022-01-07 16:55:20.712149 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:55:20.712720 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:55:20.713758 (Thread-1): Finished running node model.pro_serv_dag_auditing.stg_model_1
|
|
2022-01-07 16:55:20.714338 (Thread-1): Began running node model.pro_serv_dag_auditing.stg_model_2
|
|
2022-01-07 16:55:20.715330 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_2".
|
|
2022-01-07 16:55:20.715532 (Thread-1): Compiling model.pro_serv_dag_auditing.stg_model_2
|
|
2022-01-07 16:55:20.728722 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.stg_model_2"
|
|
2022-01-07 16:55:20.730316 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:55:20.730867 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:55:20.731642 (Thread-1): Finished running node model.pro_serv_dag_auditing.stg_model_2
|
|
2022-01-07 16:55:20.731857 (Thread-1): Began running node model.pro_serv_dag_auditing.stg_model_3
|
|
2022-01-07 16:55:20.732342 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_3".
|
|
2022-01-07 16:55:20.732790 (Thread-1): Compiling model.pro_serv_dag_auditing.stg_model_3
|
|
2022-01-07 16:55:20.745037 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.stg_model_3"
|
|
2022-01-07 16:55:20.745633 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:55:20.746005 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:55:20.747162 (Thread-1): Finished running node model.pro_serv_dag_auditing.stg_model_3
|
|
2022-01-07 16:55:20.747456 (Thread-1): Began running node model.pro_serv_dag_auditing.int_model_4
|
|
2022-01-07 16:55:20.747825 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.int_model_4".
|
|
2022-01-07 16:55:20.748255 (Thread-1): Compiling model.pro_serv_dag_auditing.int_model_4
|
|
2022-01-07 16:55:20.762468 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.int_model_4"
|
|
2022-01-07 16:55:20.764440 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:55:20.765173 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:55:20.765950 (Thread-1): Finished running node model.pro_serv_dag_auditing.int_model_4
|
|
2022-01-07 16:55:20.766292 (Thread-1): Began running node model.pro_serv_dag_auditing.fct_model_6
|
|
2022-01-07 16:55:20.766674 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.fct_model_6".
|
|
2022-01-07 16:55:20.767226 (Thread-1): Compiling model.pro_serv_dag_auditing.fct_model_6
|
|
2022-01-07 16:55:20.781765 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.fct_model_6"
|
|
2022-01-07 16:55:20.783712 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:55:20.784451 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:55:20.786299 (Thread-1): Finished running node model.pro_serv_dag_auditing.fct_model_6
|
|
2022-01-07 16:55:20.786857 (Thread-1): Began running node model.pro_serv_dag_auditing.int_model_5
|
|
2022-01-07 16:55:20.787983 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.int_model_5".
|
|
2022-01-07 16:55:20.788397 (Thread-1): Compiling model.pro_serv_dag_auditing.int_model_5
|
|
2022-01-07 16:55:20.800244 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.int_model_5"
|
|
2022-01-07 16:55:20.800767 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:55:20.801070 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:55:20.801662 (Thread-1): Finished running node model.pro_serv_dag_auditing.int_model_5
|
|
2022-01-07 16:55:20.802171 (Thread-1): Began running node model.pro_serv_dag_auditing.dim_model_7
|
|
2022-01-07 16:55:20.802651 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.dim_model_7".
|
|
2022-01-07 16:55:20.802793 (Thread-1): Compiling model.pro_serv_dag_auditing.dim_model_7
|
|
2022-01-07 16:55:20.813308 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.dim_model_7"
|
|
2022-01-07 16:55:20.813825 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:55:20.814203 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:55:20.814710 (Thread-1): Finished running node model.pro_serv_dag_auditing.dim_model_7
|
|
2022-01-07 16:55:20.815921 (MainThread): Connection 'master' was properly closed.
|
|
2022-01-07 16:55:20.816167 (MainThread): Connection 'model.pro_serv_dag_auditing.dim_model_7' was properly closed.
|
|
2022-01-07 16:55:20.843847 (MainThread): 10:55:20 | Done.
|
|
2022-01-07 16:55:20.848458 (MainThread): Acquiring new snowflake connection "generate_catalog".
|
|
2022-01-07 16:55:20.848761 (MainThread): 10:55:20 | Building catalog
|
|
2022-01-07 16:55:20.909236 (ThreadPoolExecutor-1_0): Acquiring new snowflake connection "analytics.information_schema".
|
|
2022-01-07 16:55:20.924230 (ThreadPoolExecutor-1_0): Using snowflake connection "analytics.information_schema".
|
|
2022-01-07 16:55:20.924419 (ThreadPoolExecutor-1_0): On analytics.information_schema: /* {"app": "dbt", "dbt_version": "0.18.1", "profile_name": "dbt-learn", "target_name": "dev", "connection_name": "analytics.information_schema"} */
|
|
|
|
|
|
with tables as (
|
|
|
|
select
|
|
table_catalog as "table_database",
|
|
table_schema as "table_schema",
|
|
table_name as "table_name",
|
|
table_type as "table_type",
|
|
comment as "table_comment",
|
|
|
|
-- note: this is the _role_ that owns the table
|
|
table_owner as "table_owner",
|
|
|
|
'Clustering Key' as "stats:clustering_key:label",
|
|
clustering_key as "stats:clustering_key:value",
|
|
'The key used to cluster this table' as "stats:clustering_key:description",
|
|
(clustering_key is not null) as "stats:clustering_key:include",
|
|
|
|
'Row Count' as "stats:row_count:label",
|
|
row_count as "stats:row_count:value",
|
|
'An approximate count of rows in this table' as "stats:row_count:description",
|
|
(row_count is not null) as "stats:row_count:include",
|
|
|
|
'Approximate Size' as "stats:bytes:label",
|
|
bytes as "stats:bytes:value",
|
|
'Approximate size of the table as reported by Snowflake' as "stats:bytes:description",
|
|
(bytes is not null) as "stats:bytes:include",
|
|
|
|
'Last Modified' as "stats:last_modified:label",
|
|
to_varchar(convert_timezone('UTC', last_altered), 'yyyy-mm-dd HH24:MI'||'UTC') as "stats:last_modified:value",
|
|
'The timestamp for last update/change' as "stats:last_modified:description",
|
|
(last_altered is not null and table_type='BASE TABLE') as "stats:last_modified:include"
|
|
|
|
from analytics.INFORMATION_SCHEMA.tables
|
|
|
|
),
|
|
|
|
columns as (
|
|
|
|
select
|
|
table_catalog as "table_database",
|
|
table_schema as "table_schema",
|
|
table_name as "table_name",
|
|
|
|
column_name as "column_name",
|
|
ordinal_position as "column_index",
|
|
data_type as "column_type",
|
|
comment as "column_comment"
|
|
|
|
from analytics.INFORMATION_SCHEMA.columns
|
|
)
|
|
|
|
select *
|
|
from tables
|
|
join columns using ("table_database", "table_schema", "table_name")
|
|
where (upper("table_schema") = upper('dbt_dconnors'))
|
|
order by "column_index"
|
|
2022-01-07 16:55:20.924558 (ThreadPoolExecutor-1_0): Opening a new connection, currently in state init
|
|
2022-01-07 16:55:25.033270 (ThreadPoolExecutor-1_0): SQL status: SUCCESS 228 in 4.11 seconds
|
|
2022-01-07 16:55:25.082275 (ThreadPoolExecutor-1_0): On analytics.information_schema: Close
|
|
2022-01-07 16:55:25.268769 (ThreadPoolExecutor-1_0): Acquiring new snowflake connection "real_database.information_schema".
|
|
2022-01-07 16:55:25.272611 (ThreadPoolExecutor-1_0): Using snowflake connection "real_database.information_schema".
|
|
2022-01-07 16:55:25.272739 (ThreadPoolExecutor-1_0): On real_database.information_schema: /* {"app": "dbt", "dbt_version": "0.18.1", "profile_name": "dbt-learn", "target_name": "dev", "connection_name": "real_database.information_schema"} */
|
|
|
|
|
|
with tables as (
|
|
|
|
select
|
|
table_catalog as "table_database",
|
|
table_schema as "table_schema",
|
|
table_name as "table_name",
|
|
table_type as "table_type",
|
|
comment as "table_comment",
|
|
|
|
-- note: this is the _role_ that owns the table
|
|
table_owner as "table_owner",
|
|
|
|
'Clustering Key' as "stats:clustering_key:label",
|
|
clustering_key as "stats:clustering_key:value",
|
|
'The key used to cluster this table' as "stats:clustering_key:description",
|
|
(clustering_key is not null) as "stats:clustering_key:include",
|
|
|
|
'Row Count' as "stats:row_count:label",
|
|
row_count as "stats:row_count:value",
|
|
'An approximate count of rows in this table' as "stats:row_count:description",
|
|
(row_count is not null) as "stats:row_count:include",
|
|
|
|
'Approximate Size' as "stats:bytes:label",
|
|
bytes as "stats:bytes:value",
|
|
'Approximate size of the table as reported by Snowflake' as "stats:bytes:description",
|
|
(bytes is not null) as "stats:bytes:include",
|
|
|
|
'Last Modified' as "stats:last_modified:label",
|
|
to_varchar(convert_timezone('UTC', last_altered), 'yyyy-mm-dd HH24:MI'||'UTC') as "stats:last_modified:value",
|
|
'The timestamp for last update/change' as "stats:last_modified:description",
|
|
(last_altered is not null and table_type='BASE TABLE') as "stats:last_modified:include"
|
|
|
|
from real_database.INFORMATION_SCHEMA.tables
|
|
|
|
),
|
|
|
|
columns as (
|
|
|
|
select
|
|
table_catalog as "table_database",
|
|
table_schema as "table_schema",
|
|
table_name as "table_name",
|
|
|
|
column_name as "column_name",
|
|
ordinal_position as "column_index",
|
|
data_type as "column_type",
|
|
comment as "column_comment"
|
|
|
|
from real_database.INFORMATION_SCHEMA.columns
|
|
)
|
|
|
|
select *
|
|
from tables
|
|
join columns using ("table_database", "table_schema", "table_name")
|
|
where (upper("table_schema") = upper('real_schema'))
|
|
order by "column_index"
|
|
2022-01-07 16:55:25.272842 (ThreadPoolExecutor-1_0): Opening a new connection, currently in state closed
|
|
2022-01-07 16:55:26.238858 (ThreadPoolExecutor-1_0): Snowflake query id: 01a17957-0600-fdc6-0007-978301785a12
|
|
2022-01-07 16:55:26.239051 (ThreadPoolExecutor-1_0): Snowflake error: 002003 (02000): SQL compilation error:
|
|
Database 'REAL_DATABASE' does not exist or not authorized.
|
|
2022-01-07 16:55:26.239215 (ThreadPoolExecutor-1_0): Error running SQL: macro get_catalog
|
|
2022-01-07 16:55:26.239308 (ThreadPoolExecutor-1_0): Rolling back transaction.
|
|
2022-01-07 16:55:26.239443 (ThreadPoolExecutor-1_0): On real_database.information_schema: Close
|
|
2022-01-07 16:55:26.413559 (MainThread): Encountered an error while generating catalog: Database Error
|
|
002003 (02000): SQL compilation error:
|
|
Database 'REAL_DATABASE' does not exist or not authorized.
|
|
2022-01-07 16:55:26.603551 (MainThread): dbt encountered 1 failure while writing the catalog
|
|
2022-01-07 16:55:26.603840 (MainThread): 10:55:26 | Catalog written to /Users/daveconnors/dev/proserv/pro-serv-dag-auditing/target/catalog.json
|
|
2022-01-07 16:55:26.604415 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'end', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10482d070>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10550a520>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10550a2b0>]}
|
|
2022-01-07 16:55:26.604764 (MainThread): Flushing usage events
|
|
2022-01-07 16:55:26.915088 (MainThread): Connection 'generate_catalog' was properly closed.
|
|
2022-01-07 16:55:26.915344 (MainThread): Connection 'real_database.information_schema' was properly closed.
|
|
2022-01-07 16:56:57.694835 (MainThread): Running with dbt=0.18.1
|
|
2022-01-07 16:56:57.930444 (MainThread): Loading KWallet
|
|
2022-01-07 16:56:57.931803 (MainThread): Loading SecretService
|
|
2022-01-07 16:56:57.932868 (MainThread): Loading Windows
|
|
2022-01-07 16:56:57.933936 (MainThread): Loading chainer
|
|
2022-01-07 16:56:57.934542 (MainThread): Loading macOS
|
|
2022-01-07 16:56:58.478063 (MainThread): running dbt with arguments Namespace(cls=<class 'dbt.task.generate.GenerateTask'>, compile=True, debug=False, defer=None, exclude=None, log_cache_events=False, log_format='default', models=None, partial_parse=None, profile=None, profiles_dir='/Users/daveconnors/.dbt', project_dir=None, record_timing_info=None, rpc_method='docs.generate', selector_name=None, single_threaded=False, state=None, strict=False, target=None, test_new_parser=False, threads=None, use_cache=True, use_colors=None, vars='{}', version_check=True, warn_error=False, which='generate', write_json=True)
|
|
2022-01-07 16:56:58.481371 (MainThread): Tracking: tracking
|
|
2022-01-07 16:56:58.482167 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'start', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x11217c280>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x112229670>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x1122297f0>]}
|
|
2022-01-07 16:56:58.520109 (MainThread): Partial parsing not enabled
|
|
2022-01-07 16:56:58.522362 (MainThread): Parsing macros/catalog.sql
|
|
2022-01-07 16:56:58.525911 (MainThread): Parsing macros/adapters.sql
|
|
2022-01-07 16:56:58.573507 (MainThread): Parsing macros/materializations/merge.sql
|
|
2022-01-07 16:56:58.576836 (MainThread): Parsing macros/materializations/view.sql
|
|
2022-01-07 16:56:58.578869 (MainThread): Parsing macros/materializations/table.sql
|
|
2022-01-07 16:56:58.584456 (MainThread): Parsing macros/materializations/incremental.sql
|
|
2022-01-07 16:56:58.599155 (MainThread): Parsing macros/core.sql
|
|
2022-01-07 16:56:58.604757 (MainThread): Parsing macros/materializations/helpers.sql
|
|
2022-01-07 16:56:58.617356 (MainThread): Parsing macros/materializations/snapshot/snapshot_merge.sql
|
|
2022-01-07 16:56:58.619936 (MainThread): Parsing macros/materializations/snapshot/strategies.sql
|
|
2022-01-07 16:56:58.641234 (MainThread): Parsing macros/materializations/snapshot/snapshot.sql
|
|
2022-01-07 16:56:58.676388 (MainThread): Parsing macros/materializations/seed/seed.sql
|
|
2022-01-07 16:56:58.703866 (MainThread): Parsing macros/materializations/incremental/helpers.sql
|
|
2022-01-07 16:56:58.706528 (MainThread): Parsing macros/materializations/incremental/incremental.sql
|
|
2022-01-07 16:56:58.716363 (MainThread): Parsing macros/materializations/common/merge.sql
|
|
2022-01-07 16:56:58.734795 (MainThread): Parsing macros/materializations/table/table.sql
|
|
2022-01-07 16:56:58.743116 (MainThread): Parsing macros/materializations/view/view.sql
|
|
2022-01-07 16:56:58.751164 (MainThread): Parsing macros/materializations/view/create_or_replace_view.sql
|
|
2022-01-07 16:56:58.758815 (MainThread): Parsing macros/etc/get_custom_alias.sql
|
|
2022-01-07 16:56:58.761023 (MainThread): Parsing macros/etc/query.sql
|
|
2022-01-07 16:56:58.762778 (MainThread): Parsing macros/etc/is_incremental.sql
|
|
2022-01-07 16:56:58.765363 (MainThread): Parsing macros/etc/datetime.sql
|
|
2022-01-07 16:56:58.776656 (MainThread): Parsing macros/etc/get_custom_schema.sql
|
|
2022-01-07 16:56:58.779412 (MainThread): Parsing macros/etc/get_custom_database.sql
|
|
2022-01-07 16:56:58.781760 (MainThread): Parsing macros/adapters/common.sql
|
|
2022-01-07 16:56:58.838891 (MainThread): Parsing macros/schema_tests/relationships.sql
|
|
2022-01-07 16:56:58.841605 (MainThread): Parsing macros/schema_tests/not_null.sql
|
|
2022-01-07 16:56:58.843988 (MainThread): Parsing macros/schema_tests/unique.sql
|
|
2022-01-07 16:56:58.847183 (MainThread): Parsing macros/schema_tests/accepted_values.sql
|
|
2022-01-07 16:56:58.858255 (MainThread): Partial parsing not enabled
|
|
2022-01-07 16:56:58.906398 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_3".
|
|
2022-01-07 16:56:58.931120 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_2".
|
|
2022-01-07 16:56:58.944606 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_1".
|
|
2022-01-07 16:56:58.956378 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.fct_model_6".
|
|
2022-01-07 16:56:58.969273 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.dim_model_7".
|
|
2022-01-07 16:56:58.982215 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.int_model_4".
|
|
2022-01-07 16:56:58.995999 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.int_model_5".
|
|
2022-01-07 16:56:59.009456 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.report_3".
|
|
2022-01-07 16:56:59.021943 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.report_2".
|
|
2022-01-07 16:56:59.033285 (MainThread): Acquiring new snowflake connection "model.pro_serv_dag_auditing.report_1".
|
|
2022-01-07 16:56:59.267960 (MainThread): Found 10 models, 0 tests, 0 snapshots, 0 analyses, 143 macros, 0 operations, 0 seed files, 3 sources
|
|
2022-01-07 16:56:59.269463 (MainThread):
|
|
2022-01-07 16:56:59.269950 (MainThread): Acquiring new snowflake connection "master".
|
|
2022-01-07 16:56:59.288327 (ThreadPoolExecutor-0_0): Acquiring new snowflake connection "list_analytics_dbt_dconnors".
|
|
2022-01-07 16:56:59.307763 (ThreadPoolExecutor-0_0): Using snowflake connection "list_analytics_dbt_dconnors".
|
|
2022-01-07 16:56:59.307985 (ThreadPoolExecutor-0_0): On list_analytics_dbt_dconnors: /* {"app": "dbt", "dbt_version": "0.18.1", "profile_name": "dbt-learn", "target_name": "dev", "connection_name": "list_analytics_dbt_dconnors"} */
|
|
|
|
show terse objects in analytics.dbt_dconnors
|
|
2022-01-07 16:56:59.308117 (ThreadPoolExecutor-0_0): Opening a new connection, currently in state init
|
|
2022-01-07 16:57:00.431066 (ThreadPoolExecutor-0_0): SQL status: SUCCESS 23 in 1.12 seconds
|
|
2022-01-07 16:57:00.454937 (ThreadPoolExecutor-0_0): On list_analytics_dbt_dconnors: Close
|
|
2022-01-07 16:57:00.646210 (MainThread): 10:57:00 | Concurrency: 1 threads (target='dev')
|
|
2022-01-07 16:57:00.646476 (MainThread): 10:57:00 |
|
|
2022-01-07 16:57:00.649381 (Thread-1): Began running node model.pro_serv_dag_auditing.stg_model_3
|
|
2022-01-07 16:57:00.649791 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_3".
|
|
2022-01-07 16:57:00.649942 (Thread-1): Compiling model.pro_serv_dag_auditing.stg_model_3
|
|
2022-01-07 16:57:00.673180 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.stg_model_3"
|
|
2022-01-07 16:57:00.673972 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:57:00.674255 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:57:00.674661 (Thread-1): Finished running node model.pro_serv_dag_auditing.stg_model_3
|
|
2022-01-07 16:57:00.674823 (Thread-1): Began running node model.pro_serv_dag_auditing.stg_model_1
|
|
2022-01-07 16:57:00.675072 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_1".
|
|
2022-01-07 16:57:00.675174 (Thread-1): Compiling model.pro_serv_dag_auditing.stg_model_1
|
|
2022-01-07 16:57:00.685674 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.stg_model_1"
|
|
2022-01-07 16:57:00.686615 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:57:00.687105 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:57:00.687866 (Thread-1): Finished running node model.pro_serv_dag_auditing.stg_model_1
|
|
2022-01-07 16:57:00.688087 (Thread-1): Began running node model.pro_serv_dag_auditing.stg_model_2
|
|
2022-01-07 16:57:00.688419 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.stg_model_2".
|
|
2022-01-07 16:57:00.688711 (Thread-1): Compiling model.pro_serv_dag_auditing.stg_model_2
|
|
2022-01-07 16:57:00.701805 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.stg_model_2"
|
|
2022-01-07 16:57:00.702537 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:57:00.702829 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:57:00.703298 (Thread-1): Finished running node model.pro_serv_dag_auditing.stg_model_2
|
|
2022-01-07 16:57:00.703445 (Thread-1): Began running node model.pro_serv_dag_auditing.fct_model_6
|
|
2022-01-07 16:57:00.703965 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.fct_model_6".
|
|
2022-01-07 16:57:00.704138 (Thread-1): Compiling model.pro_serv_dag_auditing.fct_model_6
|
|
2022-01-07 16:57:00.713814 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.fct_model_6"
|
|
2022-01-07 16:57:00.714411 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:57:00.714727 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:57:00.715179 (Thread-1): Finished running node model.pro_serv_dag_auditing.fct_model_6
|
|
2022-01-07 16:57:00.715333 (Thread-1): Began running node model.pro_serv_dag_auditing.int_model_4
|
|
2022-01-07 16:57:00.715609 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.int_model_4".
|
|
2022-01-07 16:57:00.715735 (Thread-1): Compiling model.pro_serv_dag_auditing.int_model_4
|
|
2022-01-07 16:57:00.726580 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.int_model_4"
|
|
2022-01-07 16:57:00.727644 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:57:00.728052 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:57:00.728518 (Thread-1): Finished running node model.pro_serv_dag_auditing.int_model_4
|
|
2022-01-07 16:57:00.728667 (Thread-1): Began running node model.pro_serv_dag_auditing.report_1
|
|
2022-01-07 16:57:00.729046 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.report_1".
|
|
2022-01-07 16:57:00.729176 (Thread-1): Compiling model.pro_serv_dag_auditing.report_1
|
|
2022-01-07 16:57:00.736835 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.report_1"
|
|
2022-01-07 16:57:00.737587 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:57:00.738132 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:57:00.738576 (Thread-1): Finished running node model.pro_serv_dag_auditing.report_1
|
|
2022-01-07 16:57:00.738712 (Thread-1): Began running node model.pro_serv_dag_auditing.report_2
|
|
2022-01-07 16:57:00.739043 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.report_2".
|
|
2022-01-07 16:57:00.739155 (Thread-1): Compiling model.pro_serv_dag_auditing.report_2
|
|
2022-01-07 16:57:00.748687 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.report_2"
|
|
2022-01-07 16:57:00.750188 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:57:00.750675 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:57:00.751363 (Thread-1): Finished running node model.pro_serv_dag_auditing.report_2
|
|
2022-01-07 16:57:00.751756 (Thread-1): Began running node model.pro_serv_dag_auditing.report_3
|
|
2022-01-07 16:57:00.752514 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.report_3".
|
|
2022-01-07 16:57:00.752951 (Thread-1): Compiling model.pro_serv_dag_auditing.report_3
|
|
2022-01-07 16:57:00.771229 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.report_3"
|
|
2022-01-07 16:57:00.773352 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:57:00.774342 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:57:00.775649 (Thread-1): Finished running node model.pro_serv_dag_auditing.report_3
|
|
2022-01-07 16:57:00.776086 (Thread-1): Began running node model.pro_serv_dag_auditing.int_model_5
|
|
2022-01-07 16:57:00.776856 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.int_model_5".
|
|
2022-01-07 16:57:00.777056 (Thread-1): Compiling model.pro_serv_dag_auditing.int_model_5
|
|
2022-01-07 16:57:00.791275 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.int_model_5"
|
|
2022-01-07 16:57:00.793030 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:57:00.793858 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:57:00.794869 (Thread-1): Finished running node model.pro_serv_dag_auditing.int_model_5
|
|
2022-01-07 16:57:00.795687 (Thread-1): Began running node model.pro_serv_dag_auditing.dim_model_7
|
|
2022-01-07 16:57:00.796327 (Thread-1): Acquiring new snowflake connection "model.pro_serv_dag_auditing.dim_model_7".
|
|
2022-01-07 16:57:00.796584 (Thread-1): Compiling model.pro_serv_dag_auditing.dim_model_7
|
|
2022-01-07 16:57:00.809721 (Thread-1): Writing injected SQL for node "model.pro_serv_dag_auditing.dim_model_7"
|
|
2022-01-07 16:57:00.810235 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:57:00.810617 (Thread-1): finished collecting timing info
|
|
2022-01-07 16:57:00.811068 (Thread-1): Finished running node model.pro_serv_dag_auditing.dim_model_7
|
|
2022-01-07 16:57:00.812073 (MainThread): Connection 'master' was properly closed.
|
|
2022-01-07 16:57:00.812366 (MainThread): Connection 'model.pro_serv_dag_auditing.dim_model_7' was properly closed.
|
|
2022-01-07 16:57:00.847219 (MainThread): 10:57:00 | Done.
|
|
2022-01-07 16:57:00.851270 (MainThread): Acquiring new snowflake connection "generate_catalog".
|
|
2022-01-07 16:57:00.851744 (MainThread): 10:57:00 | Building catalog
|
|
2022-01-07 16:57:00.937085 (ThreadPoolExecutor-1_0): Acquiring new snowflake connection "analytics.information_schema".
|
|
2022-01-07 16:57:00.948583 (ThreadPoolExecutor-1_0): Using snowflake connection "analytics.information_schema".
|
|
2022-01-07 16:57:00.948830 (ThreadPoolExecutor-1_0): On analytics.information_schema: /* {"app": "dbt", "dbt_version": "0.18.1", "profile_name": "dbt-learn", "target_name": "dev", "connection_name": "analytics.information_schema"} */
|
|
|
|
|
|
with tables as (
|
|
|
|
select
|
|
table_catalog as "table_database",
|
|
table_schema as "table_schema",
|
|
table_name as "table_name",
|
|
table_type as "table_type",
|
|
comment as "table_comment",
|
|
|
|
-- note: this is the _role_ that owns the table
|
|
table_owner as "table_owner",
|
|
|
|
'Clustering Key' as "stats:clustering_key:label",
|
|
clustering_key as "stats:clustering_key:value",
|
|
'The key used to cluster this table' as "stats:clustering_key:description",
|
|
(clustering_key is not null) as "stats:clustering_key:include",
|
|
|
|
'Row Count' as "stats:row_count:label",
|
|
row_count as "stats:row_count:value",
|
|
'An approximate count of rows in this table' as "stats:row_count:description",
|
|
(row_count is not null) as "stats:row_count:include",
|
|
|
|
'Approximate Size' as "stats:bytes:label",
|
|
bytes as "stats:bytes:value",
|
|
'Approximate size of the table as reported by Snowflake' as "stats:bytes:description",
|
|
(bytes is not null) as "stats:bytes:include",
|
|
|
|
'Last Modified' as "stats:last_modified:label",
|
|
to_varchar(convert_timezone('UTC', last_altered), 'yyyy-mm-dd HH24:MI'||'UTC') as "stats:last_modified:value",
|
|
'The timestamp for last update/change' as "stats:last_modified:description",
|
|
(last_altered is not null and table_type='BASE TABLE') as "stats:last_modified:include"
|
|
|
|
from analytics.INFORMATION_SCHEMA.tables
|
|
|
|
),
|
|
|
|
columns as (
|
|
|
|
select
|
|
table_catalog as "table_database",
|
|
table_schema as "table_schema",
|
|
table_name as "table_name",
|
|
|
|
column_name as "column_name",
|
|
ordinal_position as "column_index",
|
|
data_type as "column_type",
|
|
comment as "column_comment"
|
|
|
|
from analytics.INFORMATION_SCHEMA.columns
|
|
)
|
|
|
|
select *
|
|
from tables
|
|
join columns using ("table_database", "table_schema", "table_name")
|
|
where (upper("table_schema") = upper('dbt_dconnors'))
|
|
order by "column_index"
|
|
2022-01-07 16:57:00.948959 (ThreadPoolExecutor-1_0): Opening a new connection, currently in state init
|
|
2022-01-07 16:57:10.967016 (ThreadPoolExecutor-1_0): SQL status: SUCCESS 228 in 10.02 seconds
|
|
2022-01-07 16:57:11.039869 (ThreadPoolExecutor-1_0): On analytics.information_schema: Close
|
|
2022-01-07 16:57:11.258788 (ThreadPoolExecutor-1_0): Acquiring new snowflake connection "real_database.information_schema".
|
|
2022-01-07 16:57:11.265684 (ThreadPoolExecutor-1_0): Using snowflake connection "real_database.information_schema".
|
|
2022-01-07 16:57:11.265882 (ThreadPoolExecutor-1_0): On real_database.information_schema: /* {"app": "dbt", "dbt_version": "0.18.1", "profile_name": "dbt-learn", "target_name": "dev", "connection_name": "real_database.information_schema"} */
|
|
|
|
|
|
with tables as (
|
|
|
|
select
|
|
table_catalog as "table_database",
|
|
table_schema as "table_schema",
|
|
table_name as "table_name",
|
|
table_type as "table_type",
|
|
comment as "table_comment",
|
|
|
|
-- note: this is the _role_ that owns the table
|
|
table_owner as "table_owner",
|
|
|
|
'Clustering Key' as "stats:clustering_key:label",
|
|
clustering_key as "stats:clustering_key:value",
|
|
'The key used to cluster this table' as "stats:clustering_key:description",
|
|
(clustering_key is not null) as "stats:clustering_key:include",
|
|
|
|
'Row Count' as "stats:row_count:label",
|
|
row_count as "stats:row_count:value",
|
|
'An approximate count of rows in this table' as "stats:row_count:description",
|
|
(row_count is not null) as "stats:row_count:include",
|
|
|
|
'Approximate Size' as "stats:bytes:label",
|
|
bytes as "stats:bytes:value",
|
|
'Approximate size of the table as reported by Snowflake' as "stats:bytes:description",
|
|
(bytes is not null) as "stats:bytes:include",
|
|
|
|
'Last Modified' as "stats:last_modified:label",
|
|
to_varchar(convert_timezone('UTC', last_altered), 'yyyy-mm-dd HH24:MI'||'UTC') as "stats:last_modified:value",
|
|
'The timestamp for last update/change' as "stats:last_modified:description",
|
|
(last_altered is not null and table_type='BASE TABLE') as "stats:last_modified:include"
|
|
|
|
from real_database.INFORMATION_SCHEMA.tables
|
|
|
|
),
|
|
|
|
columns as (
|
|
|
|
select
|
|
table_catalog as "table_database",
|
|
table_schema as "table_schema",
|
|
table_name as "table_name",
|
|
|
|
column_name as "column_name",
|
|
ordinal_position as "column_index",
|
|
data_type as "column_type",
|
|
comment as "column_comment"
|
|
|
|
from real_database.INFORMATION_SCHEMA.columns
|
|
)
|
|
|
|
select *
|
|
from tables
|
|
join columns using ("table_database", "table_schema", "table_name")
|
|
where (upper("table_schema") = upper('real_schema'))
|
|
order by "column_index"
|
|
2022-01-07 16:57:11.266107 (ThreadPoolExecutor-1_0): Opening a new connection, currently in state closed
|
|
2022-01-07 16:57:12.278142 (ThreadPoolExecutor-1_0): Snowflake query id: 01a17959-0600-fe34-0007-97830178d1ca
|
|
2022-01-07 16:57:12.278415 (ThreadPoolExecutor-1_0): Snowflake error: 002003 (02000): SQL compilation error:
|
|
Database 'REAL_DATABASE' does not exist or not authorized.
|
|
2022-01-07 16:57:12.278843 (ThreadPoolExecutor-1_0): Error running SQL: macro get_catalog
|
|
2022-01-07 16:57:12.279566 (ThreadPoolExecutor-1_0): Rolling back transaction.
|
|
2022-01-07 16:57:12.279838 (ThreadPoolExecutor-1_0): On real_database.information_schema: Close
|
|
2022-01-07 16:57:12.413892 (MainThread): Encountered an error while generating catalog: Database Error
|
|
002003 (02000): SQL compilation error:
|
|
Database 'REAL_DATABASE' does not exist or not authorized.
|
|
2022-01-07 16:57:12.850213 (MainThread): dbt encountered 1 failure while writing the catalog
|
|
2022-01-07 16:57:12.850863 (MainThread): 10:57:12 | Catalog written to /Users/daveconnors/dev/proserv/pro-serv-dag-auditing/target/catalog.json
|
|
2022-01-07 16:57:12.851907 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'end', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x11217c280>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x113041b50>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x1130410d0>]}
|
|
2022-01-07 16:57:12.852620 (MainThread): Flushing usage events
|
|
2022-01-07 16:57:13.316364 (MainThread): Connection 'generate_catalog' was properly closed.
|
|
2022-01-07 16:57:13.316689 (MainThread): Connection 'real_database.information_schema' was properly closed.
|