Bumping version to 1.2.0rc2 (#5497)

* Bumping version to 1.2.0rc2

* Removing whitespace

* Update Dockerfile

* Changelog update

Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com>
Co-authored-by: Leah Antkiewicz <leah.antkiewicz@fishtownanalytics.com>

.bumpversion.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 1.2.0b1
+current_version = 1.2.0rc2
 parse = (?P<major>\d+)
 	\.(?P<minor>\d+)
 	\.(?P<patch>\d+)

.changes/1.2.0-rc1.md

@@ -0,0 +1,40 @@
+## dbt-core 1.2.0-rc1 - July 11, 2022
+### Features
+- Allow customizing `target-path` and `log-path` through environment variables and CLI flags. ([#5399](https://github.com/dbt-labs/dbt-core/issues/5399), [#5402](https://github.com/dbt-labs/dbt-core/pull/5402))
+- Move type_* macros from dbt-utils into dbt-core, with tests ([#5317](https://github.com/dbt-labs/dbt-core/issues/5317), [#5428](https://github.com/dbt-labs/dbt-core/pull/5428))
+- Add support for ratio metrics ([#4884](https://github.com/dbt-labs/dbt-core/issues/4884), [#5027](https://github.com/dbt-labs/dbt-core/pull/5027))
+- Allow users to define grants as a reasonable default in the dbt_project.yml or within each model sql or yml file combined. ([#5263](https://github.com/dbt-labs/dbt-core/issues/5263), [#5369](https://github.com/dbt-labs/dbt-core/pull/5369))
+### Fixes
+- Add inheritance to materialization macro resolution ([#4646](https://github.com/dbt-labs/dbt-core/issues/4646), [#5348](https://github.com/dbt-labs/dbt-core/pull/5348))
+- Improve pluralizations for Documentation and SqlOperation NodeTypes ([#5352](https://github.com/dbt-labs/dbt-core/issues/5352), [#5356](https://github.com/dbt-labs/dbt-core/pull/5356))
+- Properly use quotes for Snowflake snapshots when checking all columns ([#2975](https://github.com/dbt-labs/dbt-core/issues/2975), [#5389](https://github.com/dbt-labs/dbt-core/pull/5389))
+- fixes handling of RESET color code with USE_COLORS=False ([#5288](https://github.com/dbt-labs/dbt-core/issues/5288), [#5394](https://github.com/dbt-labs/dbt-core/pull/5394))
+### Docs
+- Fixed sample SQL Code for sources when no database is defined ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
+- Add support for `file:` selector in DAG viz ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
+- [Snyk] Upgrade prismjs from 1.27.0 to 1.28.0 ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
+- Run build and tests in CI checks ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
+- Improve metrics DAG viz and documentation page ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
+- Upgrade cytoscape.js fork ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
+### Under the Hood
+- Added the suggested RegEx to check the SemVer string within a package dependency and improved invalid version error handling. ([#5201](https://github.com/dbt-labs/dbt-core/issues/5201), [#5370](https://github.com/dbt-labs/dbt-core/pull/5370))
+- Add annotation to render_value method reimplemented in #5334 ([#4796](https://github.com/dbt-labs/dbt-core/issues/4796), [#5382](https://github.com/dbt-labs/dbt-core/pull/5382))
+- Bump manifest version to v6 ([#5417](https://github.com/dbt-labs/dbt-core/issues/5417), [#5430](https://github.com/dbt-labs/dbt-core/pull/5430))
+- Add tests for SQL grants ([#5437](https://github.com/dbt-labs/dbt-core/issues/5437), [#5447](https://github.com/dbt-labs/dbt-core/pull/5447))
+### Dependencies
+- Bump mypy from 0.942 to 0.961 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5337](https://github.com/dbt-labs/dbt-core/pull/5337))
+- Update colorama requirement from <0.4.5,>=0.3.9 to >=0.3.9,<0.4.6 in /core ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5388](https://github.com/dbt-labs/dbt-core/pull/5388))
+- Bump black from 22.3.0 to 22.6.0 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5420](https://github.com/dbt-labs/dbt-core/pull/5420))
+### Security
+- Move string interpolation of "secret" env vars outside of Jinja context. Update "contexts" README ([#4796](https://github.com/dbt-labs/dbt-core/issues/4796), [#5334](https://github.com/dbt-labs/dbt-core/pull/5334))
+
+### Contributors
+- [@b-per](https://github.com/b-per) ([#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
+- [@callum-mcdata](https://github.com/callum-mcdata) ([#5027](https://github.com/dbt-labs/dbt-core/pull/5027), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
+- [@darin-reify](https://github.com/darin-reify) ([#5394](https://github.com/dbt-labs/dbt-core/pull/5394))
+- [@drewbanin](https://github.com/drewbanin) ([#5027](https://github.com/dbt-labs/dbt-core/pull/5027), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
+- [@fivetran-joemarkiewicz](https://github.com/fivetran-joemarkiewicz) ([#5370](https://github.com/dbt-labs/dbt-core/pull/5370))
+- [@isidentical](https://github.com/isidentical) ([#5402](https://github.com/dbt-labs/dbt-core/pull/5402))
+- [@pdebelak](https://github.com/pdebelak) ([#5356](https://github.com/dbt-labs/dbt-core/pull/5356))
+- [@pquadri](https://github.com/pquadri) ([#5389](https://github.com/dbt-labs/dbt-core/pull/5389))
+- [@volkangurel](https://github.com/volkangurel) ([#5348](https://github.com/dbt-labs/dbt-core/pull/5348))

.changes/1.2.0-rc2.md

@@ -0,0 +1,9 @@
+## dbt-core 1.2.0-rc2 - July 20, 2022
+### Features
+- Add reusable function for retrying adapter connections. Utilize said function to add retries for Postgres (and Redshift). ([#5022](https://github.com/dbt-labs/dbt-core/issues/5022), [#5432](https://github.com/dbt-labs/dbt-core/pull/5432))
+### Fixes
+- Rename try to strict for more intuitiveness ([#5475](https://github.com/dbt-labs/dbt-core/issues/5475), [#5477](https://github.com/dbt-labs/dbt-core/pull/5477))
+
+### Contributors
+- [@jeremyyeo](https://github.com/jeremyyeo) ([#5477](https://github.com/dbt-labs/dbt-core/pull/5477))
+- [@tomasfarias](https://github.com/tomasfarias) ([#5432](https://github.com/dbt-labs/dbt-core/pull/5432))

.changes/1.2.0/Features-20220715-035555.yaml

@@ -0,0 +1,8 @@
+kind: Features
+body: Add reusable function for retrying adapter connections. Utilize said function
+  to add retries for Postgres (and Redshift).
+time: 2022-07-15T03:55:55.270637265+02:00
+custom:
+  Author: tomasfarias
+  Issue: "5022"
+  PR: "5432"

.changes/1.2.0/Fixes-20220715-231148.yaml

@@ -0,0 +1,7 @@
+kind: Fixes
+body: Rename try to strict for more intuitiveness
+time: 2022-07-15T23:11:48.327928+12:00
+custom:
+  Author: jeremyyeo
+  Issue: "5475"
+  PR: "5477"

CHANGELOG.md

@@ -6,6 +6,59 @@
 - Do not edit this file directly. This file is auto-generated using [changie](https://github.com/miniscruff/changie). For details on how to document a change, see [the contributing guide](https://github.com/dbt-labs/dbt-core/blob/main/CONTRIBUTING.md#adding-changelog-entry)
 
 
+## dbt-core 1.2.0-rc2 - July 20, 2022
+### Features
+- Add reusable function for retrying adapter connections. Utilize said function to add retries for Postgres (and Redshift). ([#5022](https://github.com/dbt-labs/dbt-core/issues/5022), [#5432](https://github.com/dbt-labs/dbt-core/pull/5432))
+### Fixes
+- Rename try to strict for more intuitiveness ([#5475](https://github.com/dbt-labs/dbt-core/issues/5475), [#5477](https://github.com/dbt-labs/dbt-core/pull/5477))
+
+### Contributors
+- [@jeremyyeo](https://github.com/jeremyyeo) ([#5477](https://github.com/dbt-labs/dbt-core/pull/5477))
+- [@tomasfarias](https://github.com/tomasfarias) ([#5432](https://github.com/dbt-labs/dbt-core/pull/5432))
+
+
+## dbt-core 1.2.0-rc1 - July 11, 2022
+### Features
+- Allow customizing `target-path` and `log-path` through environment variables and CLI flags. ([#5399](https://github.com/dbt-labs/dbt-core/issues/5399), [#5402](https://github.com/dbt-labs/dbt-core/pull/5402))
+- Move type_* macros from dbt-utils into dbt-core, with tests ([#5317](https://github.com/dbt-labs/dbt-core/issues/5317), [#5428](https://github.com/dbt-labs/dbt-core/pull/5428))
+- Add support for ratio metrics ([#4884](https://github.com/dbt-labs/dbt-core/issues/4884), [#5027](https://github.com/dbt-labs/dbt-core/pull/5027))
+- Allow users to define grants as a reasonable default in the dbt_project.yml or within each model sql or yml file combined. ([#5263](https://github.com/dbt-labs/dbt-core/issues/5263), [#5369](https://github.com/dbt-labs/dbt-core/pull/5369))
+### Fixes
+- Add inheritance to materialization macro resolution ([#4646](https://github.com/dbt-labs/dbt-core/issues/4646), [#5348](https://github.com/dbt-labs/dbt-core/pull/5348))
+- Improve pluralizations for Documentation and SqlOperation NodeTypes ([#5352](https://github.com/dbt-labs/dbt-core/issues/5352), [#5356](https://github.com/dbt-labs/dbt-core/pull/5356))
+- Properly use quotes for Snowflake snapshots when checking all columns ([#2975](https://github.com/dbt-labs/dbt-core/issues/2975), [#5389](https://github.com/dbt-labs/dbt-core/pull/5389))
+- fixes handling of RESET color code with USE_COLORS=False ([#5288](https://github.com/dbt-labs/dbt-core/issues/5288), [#5394](https://github.com/dbt-labs/dbt-core/pull/5394))
+### Docs
+- Fixed sample SQL Code for sources when no database is defined ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
+- Add support for `file:` selector in DAG viz ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
+- [Snyk] Upgrade prismjs from 1.27.0 to 1.28.0 ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
+- Run build and tests in CI checks ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
+- Improve metrics DAG viz and documentation page ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
+- Upgrade cytoscape.js fork ([#5255](https://github.com/dbt-labs/dbt-core/issues/5255), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
+### Under the Hood
+- Added the suggested RegEx to check the SemVer string within a package dependency and improved invalid version error handling. ([#5201](https://github.com/dbt-labs/dbt-core/issues/5201), [#5370](https://github.com/dbt-labs/dbt-core/pull/5370))
+- Add annotation to render_value method reimplemented in #5334 ([#4796](https://github.com/dbt-labs/dbt-core/issues/4796), [#5382](https://github.com/dbt-labs/dbt-core/pull/5382))
+- Bump manifest version to v6 ([#5417](https://github.com/dbt-labs/dbt-core/issues/5417), [#5430](https://github.com/dbt-labs/dbt-core/pull/5430))
+- Add tests for SQL grants ([#5437](https://github.com/dbt-labs/dbt-core/issues/5437), [#5447](https://github.com/dbt-labs/dbt-core/pull/5447))
+### Dependencies
+- Bump mypy from 0.942 to 0.961 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5337](https://github.com/dbt-labs/dbt-core/pull/5337))
+- Update colorama requirement from <0.4.5,>=0.3.9 to >=0.3.9,<0.4.6 in /core ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5388](https://github.com/dbt-labs/dbt-core/pull/5388))
+- Bump black from 22.3.0 to 22.6.0 ([#4904](https://github.com/dbt-labs/dbt-core/issues/4904), [#5420](https://github.com/dbt-labs/dbt-core/pull/5420))
+### Security
+- Move string interpolation of "secret" env vars outside of Jinja context. Update "contexts" README ([#4796](https://github.com/dbt-labs/dbt-core/issues/4796), [#5334](https://github.com/dbt-labs/dbt-core/pull/5334))
+
+### Contributors
+- [@b-per](https://github.com/b-per) ([#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
+- [@callum-mcdata](https://github.com/callum-mcdata) ([#5027](https://github.com/dbt-labs/dbt-core/pull/5027), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
+- [@darin-reify](https://github.com/darin-reify) ([#5394](https://github.com/dbt-labs/dbt-core/pull/5394))
+- [@drewbanin](https://github.com/drewbanin) ([#5027](https://github.com/dbt-labs/dbt-core/pull/5027), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446), [#5446](https://github.com/dbt-labs/dbt-core/pull/5446))
+- [@fivetran-joemarkiewicz](https://github.com/fivetran-joemarkiewicz) ([#5370](https://github.com/dbt-labs/dbt-core/pull/5370))
+- [@isidentical](https://github.com/isidentical) ([#5402](https://github.com/dbt-labs/dbt-core/pull/5402))
+- [@pdebelak](https://github.com/pdebelak) ([#5356](https://github.com/dbt-labs/dbt-core/pull/5356))
+- [@pquadri](https://github.com/pquadri) ([#5389](https://github.com/dbt-labs/dbt-core/pull/5389))
+- [@volkangurel](https://github.com/volkangurel) ([#5348](https://github.com/dbt-labs/dbt-core/pull/5348))
+
+
 ## dbt-core 1.2.0-b1 - June 24, 2022
 ### Features
 - Add selector method when reading selector definitions ([#4821](https://github.com/dbt-labs/dbt-core/issues/4821), [#4827](https://github.com/dbt-labs/dbt-core/pull/4827))

core/dbt/adapters/base/connections.py

@@ -1,10 +1,24 @@
 import abc
 import os
+from time import sleep
+import sys
 
 # multiprocessing.RLock is a function returning this type
 from multiprocessing.synchronize import RLock
 from threading import get_ident
-from typing import Dict, Tuple, Hashable, Optional, ContextManager, List
+from typing import (
+    Any,
+    Dict,
+    Tuple,
+    Hashable,
+    Optional,
+    ContextManager,
+    List,
+    Type,
+    Union,
+    Iterable,
+    Callable,
+)
 
 import agate
 
@@ -21,6 +35,7 @@ from dbt.contracts.graph.manifest import Manifest
 from dbt.adapters.base.query_headers import (
     MacroQueryStringSetter,
 )
+from dbt.events import AdapterLogger
 from dbt.events.functions import fire_event
 from dbt.events.types import (
     NewConnection,
@@ -34,6 +49,9 @@ from dbt.events.types import (
 )
 from dbt import flags
 
+SleepTime = Union[int, float]  # As taken by time.sleep.
+AdapterHandle = Any  # Adapter connection handle objects can be any class.
+
 
 class BaseConnectionManager(metaclass=abc.ABCMeta):
     """Methods to implement:
@@ -159,6 +177,94 @@ class BaseConnectionManager(metaclass=abc.ABCMeta):
         conn.name = conn_name
         return conn
 
+    @classmethod
+    def retry_connection(
+        cls,
+        connection: Connection,
+        connect: Callable[[], AdapterHandle],
+        logger: AdapterLogger,
+        retryable_exceptions: Iterable[Type[Exception]],
+        retry_limit: int = 1,
+        retry_timeout: Union[Callable[[int], SleepTime], SleepTime] = 1,
+        _attempts: int = 0,
+    ) -> Connection:
+        """Given a Connection, set its handle by calling connect.
+
+        The calls to connect will be retried up to retry_limit times to deal with transient
+        connection errors. By default, one retry will be attempted if retryable_exceptions is set.
+
+        :param Connection connection: An instance of a Connection that needs a handle to be set,
+            usually when attempting to open it.
+        :param connect: A callable that returns the appropiate connection handle for a
+            given adapter. This callable will be retried retry_limit times if a subclass of any
+            Exception in retryable_exceptions is raised by connect.
+        :type connect: Callable[[], AdapterHandle]
+        :param AdapterLogger logger: A logger to emit messages on retry attempts or errors. When
+            handling expected errors, we call debug, and call warning on unexpected errors or when
+            all retry attempts have been exhausted.
+        :param retryable_exceptions: An iterable of exception classes that if raised by
+            connect should trigger a retry.
+        :type retryable_exceptions: Iterable[Type[Exception]]
+        :param int retry_limit: How many times to retry the call to connect. If this limit
+            is exceeded before a successful call, a FailedToConnectException will be raised.
+            Must be non-negative.
+        :param retry_timeout: Time to wait between attempts to connect. Can also take a
+            Callable that takes the number of attempts so far, beginning at 0, and returns an int
+            or float to be passed to time.sleep.
+        :type retry_timeout: Union[Callable[[int], SleepTime], SleepTime] = 1
+        :param int _attempts: Parameter used to keep track of the number of attempts in calling the
+            connect function across recursive calls. Passed as an argument to retry_timeout if it
+            is a Callable. This parameter should not be set by the initial caller.
+        :raises dbt.exceptions.FailedToConnectException: Upon exhausting all retry attempts without
+            successfully acquiring a handle.
+        :return: The given connection with its appropriate state and handle attributes set
+            depending on whether we successfully acquired a handle or not.
+        """
+        timeout = retry_timeout(_attempts) if callable(retry_timeout) else retry_timeout
+        if timeout < 0:
+            raise dbt.exceptions.FailedToConnectException(
+                "retry_timeout cannot be negative or return a negative time."
+            )
+
+        if retry_limit < 0 or retry_limit > sys.getrecursionlimit():
+            # This guard is not perfect others may add to the recursion limit (e.g. built-ins).
+            connection.handle = None
+            connection.state = ConnectionState.FAIL
+            raise dbt.exceptions.FailedToConnectException("retry_limit cannot be negative")
+
+        try:
+            connection.handle = connect()
+            connection.state = ConnectionState.OPEN
+            return connection
+
+        except tuple(retryable_exceptions) as e:
+            if retry_limit <= 0:
+                connection.handle = None
+                connection.state = ConnectionState.FAIL
+                raise dbt.exceptions.FailedToConnectException(str(e))
+
+            logger.debug(
+                f"Got a retryable error when attempting to open a {cls.TYPE} connection.\n"
+                f"{retry_limit} attempts remaining. Retrying in {timeout} seconds.\n"
+                f"Error:\n{e}"
+            )
+
+            sleep(timeout)
+            return cls.retry_connection(
+                connection=connection,
+                connect=connect,
+                logger=logger,
+                retry_limit=retry_limit - 1,
+                retry_timeout=retry_timeout,
+                retryable_exceptions=retryable_exceptions,
+                _attempts=_attempts + 1,
+            )
+
+        except Exception as e:
+            connection.handle = None
+            connection.state = ConnectionState.FAIL
+            raise dbt.exceptions.FailedToConnectException(str(e))
+
     @abc.abstractmethod
     def cancel_open(self) -> Optional[List[str]]:
         """Cancel all open connections on the adapter. (passable)"""

core/dbt/context/base.py

@@ -474,19 +474,17 @@ class BaseContext(metaclass=ContextMeta):
 
     @contextmember
     @staticmethod
-    def try_set(value: Iterable[Any]) -> Set[Any]:
-        """The `try_set` context method can be used to convert any iterable
+    def set_strict(value: Iterable[Any]) -> Set[Any]:
+        """The `set_strict` context method can be used to convert any iterable
         to a sequence of iterable elements that are unique (a set). The
-        difference to the `set` context method is that the `try_set` method
+        difference to the `set` context method is that the `set_strict` method
         will raise an exception on a TypeError.
 
         :param value: The iterable
-        :param default: A default value to return if the `value` argument
-            is not an iterable
 
         Usage:
             {% set my_list = [1, 2, 2, 3] %}
-            {% set my_set = try_set(my_list) %}
+            {% set my_set = set_strict(my_list) %}
             {% do log(my_set) %}  {# {1, 2, 3} #}
         """
         try:
@@ -497,7 +495,7 @@ class BaseContext(metaclass=ContextMeta):
     @contextmember("zip")
     @staticmethod
     def _zip(*args: Iterable[Any], default: Any = None) -> Optional[Iterable[Any]]:
-        """The `try_zip` context method can be used to used to return
+        """The `zip` context method can be used to used to return
         an iterator of tuples, where the i-th tuple contains the i-th
         element from each of the argument iterables.
 
@@ -518,21 +516,19 @@ class BaseContext(metaclass=ContextMeta):
 
     @contextmember
     @staticmethod
-    def try_zip(*args: Iterable[Any]) -> Iterable[Any]:
-        """The `try_zip` context method can be used to used to return
+    def zip_strict(*args: Iterable[Any]) -> Iterable[Any]:
+        """The `zip_strict` context method can be used to used to return
         an iterator of tuples, where the i-th tuple contains the i-th
         element from each of the argument iterables. The difference to the
-        `zip` context method is that the `try_zip` method will raise an
+        `zip` context method is that the `zip_strict` method will raise an
         exception on a TypeError.
 
         :param *args: Any number of iterables
-        :param default: A default value to return if `*args` is not
-            iterable
 
         Usage:
             {% set my_list_a = [1, 2] %}
             {% set my_list_b = ['alice', 'bob'] %}
-            {% set my_zip = try_zip(my_list_a, my_list_b) | list %}
+            {% set my_zip = zip_strict(my_list_a, my_list_b) | list %}
             {% do log(my_set) %}  {# [(1, 'alice'), (2, 'bob')] #}
         """
         try:

core/dbt/version.py

@@ -235,5 +235,5 @@ def _get_adapter_plugin_names() -> Iterator[str]:
             yield plugin_name
 
 
-__version__ = "1.2.0b1"
+__version__ = "1.2.0rc2"
 installed = get_installed_version()

core/setup.py

@@ -25,7 +25,7 @@ with open(os.path.join(this_directory, "README.md")) as f:
 
 
 package_name = "dbt-core"
-package_version = "1.2.0b1"
+package_version = "1.2.0rc2"
 description = """With dbt, data analysts and engineers can build analytics \
 the way engineers build applications."""
 

docker/Dockerfile

@@ -14,12 +14,12 @@ FROM --platform=$build_for python:3.10.5-slim-bullseye as base
 # N.B. The refs updated automagically every release via bumpversion
 # N.B. dbt-postgres is currently found in the core codebase so a value of dbt-core@<some_version> is correct
 
-ARG dbt_core_ref=dbt-core@v1.2.0b1
-ARG dbt_postgres_ref=dbt-core@v1.2.0b1
-ARG dbt_redshift_ref=dbt-redshift@v1.0.0
-ARG dbt_bigquery_ref=dbt-bigquery@v1.0.0
-ARG dbt_snowflake_ref=dbt-snowflake@v1.0.0
-ARG dbt_spark_ref=dbt-spark@v1.0.0
+ARG dbt_core_ref=dbt-core@v1.2.0rc2
+ARG dbt_postgres_ref=dbt-core@v1.2.0rc2
+ARG dbt_redshift_ref=dbt-redshift@v1.2.0rc1
+ARG dbt_bigquery_ref=dbt-bigquery@v1.2.0rc1
+ARG dbt_snowflake_ref=dbt-snowflake@v1.2.0rc1
+ARG dbt_spark_ref=dbt-spark@v1.2.0rc1
 # special case args
 ARG dbt_spark_version=all
 ARG dbt_third_party

plugins/postgres/dbt/adapters/postgres/__version__.py

@@ -1 +1 @@
-version = "1.2.0b1"
+version = "1.2.0rc2"

plugins/postgres/dbt/adapters/postgres/connections.py

@@ -31,6 +31,7 @@ class PostgresCredentials(Credentials):
     sslkey: Optional[str] = None
     sslrootcert: Optional[str] = None
     application_name: Optional[str] = "dbt"
+    retries: int = 1
 
     _ALIASES = {"dbname": "database", "pass": "password"}
 
@@ -121,7 +122,7 @@ class PostgresConnectionManager(SQLConnectionManager):
         if credentials.application_name:
             kwargs["application_name"] = credentials.application_name
 
-        try:
+        def connect():
             handle = psycopg2.connect(
                 dbname=credentials.database,
                 user=credentials.user,
@@ -131,23 +132,26 @@ class PostgresConnectionManager(SQLConnectionManager):
                 connect_timeout=credentials.connect_timeout,
                 **kwargs,
             )
-
             if credentials.role:
                 handle.cursor().execute("set role {}".format(credentials.role))
+            return handle
 
-            connection.handle = handle
-            connection.state = "open"
-        except psycopg2.Error as e:
-            logger.debug(
-                "Got an error when attempting to open a postgres " "connection: '{}'".format(e)
-            )
+        retryable_exceptions = [
+            # OperationalError is subclassed by all psycopg2 Connection Exceptions and it's raised
+            # by generic connection timeouts without an error code. This is a limitation of
+            # psycopg2 which doesn't provide subclasses for errors without a SQLSTATE error code.
+            # The limitation has been known for a while and there are no efforts to tackle it.
+            # See: https://github.com/psycopg/psycopg2/issues/682
+            psycopg2.errors.OperationalError,
+        ]
 
-            connection.handle = None
-            connection.state = "fail"
-
-            raise dbt.exceptions.FailedToConnectException(str(e))
-
-        return connection
+        return cls.retry_connection(
+            connection,
+            connect=connect,
+            logger=logger,
+            retry_limit=credentials.retries,
+            retryable_exceptions=retryable_exceptions,
+        )
 
     def cancel(self, connection):
         connection_name = connection.name

plugins/postgres/setup.py

@@ -41,7 +41,7 @@ def _dbt_psycopg2_name():
 
 
 package_name = "dbt-postgres"
-package_version = "1.2.0b1"
+package_version = "1.2.0rc2"
 description = """The postgres adapter plugin for dbt (data build tool)"""
 
 this_directory = os.path.abspath(os.path.dirname(__file__))

test/unit/test_adapter_connection_manager.py

@@ -0,0 +1,494 @@
+import unittest
+from unittest import mock
+import sys
+
+import dbt.exceptions
+
+import psycopg2
+
+from dbt.contracts.connection import Connection
+from dbt.adapters.base import BaseConnectionManager
+from dbt.adapters.postgres import PostgresCredentials, PostgresConnectionManager
+from dbt.events import AdapterLogger
+
+
+class BaseConnectionManagerTest(unittest.TestCase):
+    def setUp(self):
+        self.postgres_credentials = PostgresCredentials(
+            host="localhost",
+            user="test-user",
+            port=1111,
+            password="test-password",
+            database="test-db",
+            schema="test-schema",
+        )
+        self.logger = AdapterLogger("test")
+        self.postgres_connection = Connection("postgres", None, self.postgres_credentials)
+
+    def test_retry_connection(self):
+        """Test a dummy handle is set on a connection on the first attempt.
+
+        This test uses a Connection populated with test PostgresCredentials values, and
+        expects the Connection.handle attribute to be set to True and it's state to
+        "open", after calling retry_connection.
+
+        Moreover, the attribute should be set in the first attempt as no exception would
+        be raised for retrying. A mock connect function is used to simulate a real connection
+        passing on the first attempt.
+        """
+        conn = self.postgres_connection
+        attempts = 0
+
+        def connect():
+            nonlocal attempts
+            attempts += 1
+            return True
+
+        conn = BaseConnectionManager.retry_connection(
+            conn,
+            connect,
+            self.logger,
+            retryable_exceptions=[],
+        )
+
+        assert conn.state == "open"
+        assert conn.handle is True
+        assert attempts == 1
+
+    def test_retry_connection_fails_unhandled(self):
+        """Test setting a handle fails upon raising a non-handled exception.
+
+        This test uses a Connection populated with test PostgresCredentials values, and
+        expects a ValueError to be raised by a mock connect function. As a
+        result:
+        * The Connection state should be "fail" and the handle None.
+        * The resulting attempt count should be 1 as we are not explicitly configured to handle a
+          ValueError.
+        * retry_connection should raise a FailedToConnectException with the Exception message.
+        """
+        conn = self.postgres_connection
+        attempts = 0
+
+        def connect():
+            nonlocal attempts
+            attempts += 1
+            raise ValueError("Something went horribly wrong")
+
+        with self.assertRaisesRegex(
+            dbt.exceptions.FailedToConnectException, "Something went horribly wrong"
+        ):
+
+            BaseConnectionManager.retry_connection(
+                conn,
+                connect,
+                self.logger,
+                retry_limit=1,
+                retry_timeout=lambda attempt: 0,
+                retryable_exceptions=(TypeError,),
+            )
+
+        assert conn.state == "fail"
+        assert conn.handle is None
+        assert attempts == 1
+
+    def test_retry_connection_fails_handled(self):
+        """Test setting a handle fails upon raising a handled exception.
+
+        This test uses a Connection populated with test PostgresCredentials values, and
+        expects a ValueError to be raised by a mock connect function.
+        As a result:
+        * The Connection state should be "fail" and the handle None.
+        * The resulting attempt count should be 2 as we are configured to handle a ValueError.
+        * retry_connection should raise a FailedToConnectException with the Exception message.
+        """
+        conn = self.postgres_connection
+        attempts = 0
+
+        def connect():
+            nonlocal attempts
+            attempts += 1
+            raise ValueError("Something went horribly wrong")
+
+        with self.assertRaisesRegex(
+            dbt.exceptions.FailedToConnectException, "Something went horribly wrong"
+        ):
+
+            BaseConnectionManager.retry_connection(
+                conn,
+                connect,
+                self.logger,
+                retry_timeout=0,
+                retryable_exceptions=(ValueError,),
+                retry_limit=1,
+            )
+
+        assert conn.state == "fail"
+        assert conn.handle is None
+
+    def test_retry_connection_passes_handled(self):
+        """Test setting a handle fails upon raising a handled exception.
+
+        This test uses a Connection populated with test PostgresCredentials values, and
+        expects a ValueError to be raised by a mock connect function only the first
+        time is called. Upon handling the exception once, connect should return.
+        As a result:
+        * The Connection state should be "open" and the handle True.
+        * The resulting attempt count should be 2 as we are configured to handle a ValueError.
+        """
+        conn = self.postgres_connection
+        is_handled = False
+        attempts = 0
+
+        def connect():
+            nonlocal is_handled
+            nonlocal attempts
+
+            attempts += 1
+
+            if is_handled:
+                return True
+
+            is_handled = True
+            raise ValueError("Something went horribly wrong")
+
+        conn = BaseConnectionManager.retry_connection(
+            conn,
+            connect,
+            self.logger,
+            retry_timeout=0,
+            retryable_exceptions=(ValueError,),
+            retry_limit=1,
+        )
+
+        assert conn.state == "open"
+        assert conn.handle is True
+        assert is_handled is True
+        assert attempts == 2
+
+    def test_retry_connection_attempts(self):
+        """Test setting a handle fails upon raising a handled exception multiple times.
+
+        This test uses a Connection populated with test PostgresCredentials values, and
+        expects a ValueError to be raised by a mock connect function. As a result:
+        * The Connection state should be "fail" and the handle None, as connect
+          never returns.
+        * The resulting attempt count should be 11 as we are configured to handle a ValueError.
+        * retry_connection should raise a FailedToConnectException with the Exception message.
+        """
+        conn = self.postgres_connection
+        attempts = 0
+
+        def connect():
+            nonlocal attempts
+            attempts += 1
+
+            raise ValueError("Something went horribly wrong")
+
+        with self.assertRaisesRegex(
+            dbt.exceptions.FailedToConnectException, "Something went horribly wrong"
+        ):
+            BaseConnectionManager.retry_connection(
+                conn,
+                connect,
+                self.logger,
+                retry_timeout=0,
+                retryable_exceptions=(ValueError,),
+                retry_limit=10,
+            )
+
+        assert conn.state == "fail"
+        assert conn.handle is None
+        assert attempts == 11
+
+    def test_retry_connection_fails_handling_all_exceptions(self):
+        """Test setting a handle fails after exhausting all attempts.
+
+        This test uses a Connection populated with test PostgresCredentials values, and
+        expects a TypeError to be raised by a mock connect function. As a result:
+        * The Connection state should be "fail" and the handle None, as connect
+          never returns.
+        * The resulting attempt count should be 11 as we are configured to handle all Exceptions.
+        * retry_connection should raise a FailedToConnectException with the Exception message.
+        """
+        conn = self.postgres_connection
+        attempts = 0
+
+        def connect():
+            nonlocal attempts
+            attempts += 1
+
+            raise TypeError("An unhandled thing went horribly wrong")
+
+        with self.assertRaisesRegex(
+            dbt.exceptions.FailedToConnectException, "An unhandled thing went horribly wrong"
+        ):
+            BaseConnectionManager.retry_connection(
+                conn,
+                connect,
+                self.logger,
+                retry_timeout=0,
+                retryable_exceptions=[Exception],
+                retry_limit=15,
+            )
+
+        assert conn.state == "fail"
+        assert conn.handle is None
+        assert attempts == 16
+
+    def test_retry_connection_passes_multiple_handled(self):
+        """Test setting a handle passes upon handling multiple exceptions.
+
+        This test uses a Connection populated with test PostgresCredentials values, and
+        expects a mock connect to raise a ValueError in the first invocation and a
+        TypeError in the second invocation. As a result:
+        * The Connection state should be "open" and the handle True, as connect
+          returns after both exceptions have been handled.
+        * The resulting attempt count should be 3.
+        """
+        conn = self.postgres_connection
+        is_value_err_handled = False
+        is_type_err_handled = False
+        attempts = 0
+
+        def connect():
+            nonlocal is_value_err_handled
+            nonlocal is_type_err_handled
+            nonlocal attempts
+
+            attempts += 1
+
+            if is_value_err_handled and is_type_err_handled:
+                return True
+            elif is_type_err_handled:
+                is_value_err_handled = True
+                raise ValueError("Something went horribly wrong")
+            else:
+                is_type_err_handled = True
+                raise TypeError("An unhandled thing went horribly wrong")
+
+        conn = BaseConnectionManager.retry_connection(
+            conn,
+            connect,
+            self.logger,
+            retry_timeout=0,
+            retryable_exceptions=(ValueError, TypeError),
+            retry_limit=2,
+        )
+
+        assert conn.state == "open"
+        assert conn.handle is True
+        assert is_type_err_handled is True
+        assert is_value_err_handled is True
+        assert attempts == 3
+
+    def test_retry_connection_passes_none_excluded(self):
+        """Test setting a handle passes upon handling multiple exceptions.
+
+        This test uses a Connection populated with test PostgresCredentials values, and
+        expects a mock connect to raise a ValueError in the first invocation and a
+        TypeError in the second invocation. As a result:
+        * The Connection state should be "open" and the handle True, as connect
+          returns after both exceptions have been handled.
+        * The resulting attempt count should be 3.
+        """
+        conn = self.postgres_connection
+        is_value_err_handled = False
+        is_type_err_handled = False
+        attempts = 0
+
+        def connect():
+            nonlocal is_value_err_handled
+            nonlocal is_type_err_handled
+            nonlocal attempts
+
+            attempts += 1
+
+            if is_value_err_handled and is_type_err_handled:
+                return True
+            elif is_type_err_handled:
+                is_value_err_handled = True
+                raise ValueError("Something went horribly wrong")
+            else:
+                is_type_err_handled = True
+                raise TypeError("An unhandled thing went horribly wrong")
+
+        conn = BaseConnectionManager.retry_connection(
+            conn,
+            connect,
+            self.logger,
+            retry_timeout=0,
+            retryable_exceptions=(ValueError, TypeError),
+            retry_limit=2,
+        )
+
+        assert conn.state == "open"
+        assert conn.handle is True
+        assert is_type_err_handled is True
+        assert is_value_err_handled is True
+        assert attempts == 3
+
+    def test_retry_connection_retry_limit(self):
+        """Test retry_connection raises an exception with a negative retry limit."""
+        conn = self.postgres_connection
+        attempts = 0
+
+        def connect():
+            nonlocal attempts
+            attempts += 1
+            return True
+
+        with self.assertRaisesRegex(
+            dbt.exceptions.FailedToConnectException, "retry_limit cannot be negative"
+        ):
+            BaseConnectionManager.retry_connection(
+                conn,
+                connect,
+                self.logger,
+                retry_timeout=0,
+                retryable_exceptions=(ValueError,),
+                retry_limit=-2,
+            )
+
+        assert conn.state == "fail"
+        assert conn.handle is None
+        assert attempts == 0
+
+    def test_retry_connection_retry_timeout(self):
+        """Test retry_connection raises an exception with a negative timeout."""
+        conn = self.postgres_connection
+        attempts = 0
+
+        def connect():
+            nonlocal attempts
+            attempts += 1
+            return True
+
+        for retry_timeout in [-10, -2.5, lambda _: -100, lambda _: -10.1]:
+            with self.assertRaisesRegex(
+                dbt.exceptions.FailedToConnectException,
+                "retry_timeout cannot be negative or return a negative time",
+            ):
+                BaseConnectionManager.retry_connection(
+                    conn,
+                    connect,
+                    self.logger,
+                    retry_timeout=-10,
+                    retryable_exceptions=(ValueError,),
+                    retry_limit=2,
+                )
+
+        assert conn.state == "init"
+        assert conn.handle is None
+        assert attempts == 0
+
+    def test_retry_connection_exceeds_recursion_limit(self):
+        """Test retry_connection raises an exception with retries that exceed recursion limit."""
+        conn = self.postgres_connection
+        attempts = 0
+
+        def connect():
+            nonlocal attempts
+            attempts += 1
+            return True
+
+        with self.assertRaisesRegex(
+            dbt.exceptions.FailedToConnectException,
+            "retry_limit cannot be negative",
+        ):
+            BaseConnectionManager.retry_connection(
+                conn,
+                connect,
+                self.logger,
+                retry_timeout=2,
+                retryable_exceptions=(ValueError,),
+                retry_limit=sys.getrecursionlimit() + 1,
+            )
+
+        assert conn.state == "fail"
+        assert conn.handle is None
+        assert attempts == 0
+
+    def test_retry_connection_with_exponential_backoff_timeout(self):
+        """Test retry_connection with an exponential backoff timeout.
+
+        We assert the provided exponential backoff function gets passed the right attempt number
+        and produces the expected timeouts.
+        """
+        conn = self.postgres_connection
+        attempts = 0
+        timeouts = []
+
+        def connect():
+            nonlocal attempts
+            attempts += 1
+
+            if attempts < 12:
+                raise ValueError("Keep trying!")
+            return True
+
+        def exp_backoff(n):
+            nonlocal timeouts
+            computed = 2**n
+            # We store the computed values to ensure they match the expected backoff...
+            timeouts.append((n, computed))
+            # but we return 0 as we don't want the test to go on forever.
+            return 0
+
+        conn = BaseConnectionManager.retry_connection(
+            conn,
+            connect,
+            self.logger,
+            retry_timeout=exp_backoff,
+            retryable_exceptions=(ValueError,),
+            retry_limit=12,
+        )
+
+        assert conn.state == "open"
+        assert conn.handle is True
+        assert attempts == 12
+        assert timeouts == [(n, 2**n) for n in range(12)]
+
+
+class PostgresConnectionManagerTest(unittest.TestCase):
+    def setUp(self):
+        self.credentials = PostgresCredentials(
+            host="localhost",
+            user="test-user",
+            port=1111,
+            password="test-password",
+            database="test-db",
+            schema="test-schema",
+            retries=2,
+        )
+        self.connection = Connection("postgres", None, self.credentials)
+
+    def test_open(self):
+        """Test opening a Postgres Connection with failures in the first 3 attempts.
+
+        This test uses a Connection populated with test PostgresCredentials values, and
+        expects a mock connect to raise a psycopg2.errors.ConnectionFailuer
+        in the first 3 invocations, after which the mock should return True. As a result:
+        * The Connection state should be "open" and the handle True, as connect
+          returns in the 4th attempt.
+        * The resulting attempt count should be 4.
+        """
+        conn = self.connection
+        attempt = 0
+
+        def connect(*args, **kwargs):
+            nonlocal attempt
+            attempt += 1
+
+            if attempt <= 2:
+                raise psycopg2.errors.ConnectionFailure("Connection has failed")
+
+            return True
+
+        with mock.patch("psycopg2.connect", wraps=connect) as mock_connect:
+            PostgresConnectionManager.open(conn)
+
+            assert mock_connect.call_count == 3
+
+        assert attempt == 3
+        assert conn.state == "open"
+        assert conn.handle is True

test/unit/test_context.py

@@ -190,9 +190,9 @@ REQUIRED_BASE_KEYS = frozenset(
         "fromyaml",
         "toyaml",
         "set",
-        "try_set",
+        "set_strict",
         "zip",
-        "try_zip",
+        "zip_strict",
         "log",
         "run_started_at",
         "invocation_id",

tests/adapter/dbt/tests/adapter/__version__.py

@@ -1 +1 @@
-version = "1.2.0b1"
+version = "1.2.0rc2"

tests/adapter/setup.py

@@ -20,7 +20,7 @@ except ImportError:
 
 
 package_name = "dbt-tests-adapter"
-package_version = "1.2.0b1"
+package_version = "1.2.0rc2"
 description = """The dbt adapter tests for adapter plugins"""
 
 this_directory = os.path.abspath(os.path.dirname(__file__))

tests/functional/context_methods/test_builtin_functions.py

@@ -7,8 +7,8 @@ macros__validate_set_sql = """
 {% macro validate_set() %}
     {% set set_result = set([1, 2, 2, 3, 'foo', False]) %}
     {{ log("set_result: " ~ set_result) }}
-    {% set try_set_result = try_set([1, 2, 2, 3, 'foo', False]) %}
-    {{ log("try_set_result: " ~ try_set_result) }}
+    {% set set_strict_result = set_strict([1, 2, 2, 3, 'foo', False]) %}
+    {{ log("set_strict_result: " ~ set_strict_result) }}
 {% endmacro %}
 """
 
@@ -18,17 +18,17 @@ macros__validate_zip_sql = """
     {% set list_b = ['foo', 'bar'] %}
     {% set zip_result = zip(list_a, list_b) | list %}
     {{ log("zip_result: " ~ zip_result) }}
-    {% set try_zip_result = try_zip(list_a, list_b) | list %}
-    {{ log("try_zip_result: " ~ try_zip_result) }}
+    {% set zip_strict_result = zip_strict(list_a, list_b) | list %}
+    {{ log("zip_strict_result: " ~ zip_strict_result) }}
 {% endmacro %}
 """
 
 models__set_exception_sql = """
-{% set try_set_result = try_set(1) %}
+{% set set_strict_result = set_strict(1) %}
 """
 
 models__zip_exception_sql = """
-{% set try_set_result = try_zip(1) %}
+{% set zip_strict_result = zip_strict(1) %}
 """
 
 
@@ -46,18 +46,18 @@ class TestContextBuiltins:
         # The order of the set isn't guaranteed so we can't check for the actual set in the logs
         assert "set_result: " in log_output
         assert "False" in log_output
-        assert "try_set_result: " in log_output
+        assert "set_strict_result: " in log_output
 
     def test_builtin_zip_function(self, project):
         _, log_output = run_dbt_and_capture(["--debug", "run-operation", "validate_zip"])
 
         expected_zip = [(1, "foo"), (2, "bar")]
         assert f"zip_result: {expected_zip}" in log_output
-        assert f"try_zip_result: {expected_zip}" in log_output
+        assert f"zip_strict_result: {expected_zip}" in log_output
 
 
 class TestContextBuiltinExceptions:
-    # Assert compilation errors are raised with try_ equivalents
+    # Assert compilation errors are raised with _strict equivalents
     def test_builtin_function_exception(self, project):
         write_file(models__set_exception_sql, project.project_root, "models", "raise.sql")
         with pytest.raises(CompilationException):