mirror of
https://github.com/bitol-io/open-data-contract-standard.git
synced 2025-05-28 19:34:02 +00:00
Merge branch 'dev' into issue-add-tags-to-readme
This commit is contained in:
Peter Flook
GitHub
@@ -49,7 +49,7 @@ kind: DataContract
|
||||
id: 53581432-6c55-4ba2-a65f-72344a91553a
|
||||
name: seller_payments_v1
|
||||
version: 1.1.0 # Data Contract Version
|
||||
status: production
|
||||
status: active
|
||||
domain: seller
|
||||
dataProduct: payments
|
||||
tenant: ClimateQuantumInc
|
||||
@@ -64,21 +64,25 @@ tags: ['finance']
|
||||
|
||||
### Definitions
|
||||
|
||||
| Key | UX label | Required | Description |
|
||||
|-------------------------|------------------|----------|------------------------------------------------------------------------------------------------|
|
||||
| apiVersion | Standard version | Yes | Version of the standard used to build data contract. Default value is `v3.0.0`. |
|
||||
| kind | Kind | Yes | The kind of file this is. Valid value is `DataContract`. |
|
||||
| id | ID | Yes | A unique identifier used to reduce the risk of dataset name collisions, such as a UUID. |
|
||||
| name | Name | No | Name of the data contract. |
|
||||
| version | Version | Yes | Current version of the data contract. |
|
||||
| status | Status | Yes | Current status of the data contract. Valid values are `production`, `test`, or `development`. |
|
||||
| tenant | Tenant | No | Indicates the property the data is primarily associated with. Value is case insensitive. |
|
||||
| domain | Domain | No | Name of the logical data domain. |
|
||||
| dataProduct | Data Product | No | Name of the data product. |
|
||||
| description | Description | No | Object containing the descriptions. |
|
||||
| description.purpose | Purpose | No | Intended purpose for the provided data. |
|
||||
| description.limitations | Limitations | No | Technical, compliance, and legal limitations for data use. |
|
||||
| description.usage | Usage | No | Recommended usage of the data. |
|
||||
| Key | UX label | Required | Description |
|
||||
|--------------------------------------|---------------------------|----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| apiVersion | Standard version | Yes | Version of the standard used to build data contract. Default value is `v3.0.0`. |
|
||||
| kind | Kind | Yes | The kind of file this is. Valid value is `DataContract`. |
|
||||
| id | ID | Yes | A unique identifier used to reduce the risk of dataset name collisions, such as a UUID. |
|
||||
| name | Name | No | Name of the data contract. |
|
||||
| version | Version | Yes | Current version of the data contract. |
|
||||
| status | Status | Yes | Current status of the data contract. Examples are "proposed", "draft", "active", "deprecated", "retired". |
|
||||
| tenant | Tenant | No | Indicates the property the data is primarily associated with. Value is case insensitive. |
|
||||
| domain | Domain | No | Name of the logical data domain. |
|
||||
| dataProduct | Data Product | No | Name of the data product. |
|
||||
| authoritativeDefinitions | Authoritative Definitions | No | List of links to sources that provide more details on the data contract. |
|
||||
| description | Description | No | Object containing the descriptions. |
|
||||
| description.purpose | Purpose | No | Intended purpose for the provided data. |
|
||||
| description.limitations | Limitations | No | Technical, compliance, and legal limitations for data use. |
|
||||
| description.usage | Usage | No | Recommended usage of the data. |
|
||||
| description.authoritativeDefinitions | Authoritative Definitions | No | List of links to sources that provide more details on the dataset; examples would be a link to privacy statement, terms and conditions, license agreements, data catalog, or another tool. |
|
||||
| description.customProperties | Custom Properties | No | Custom properties that are not part of the standard. |
|
||||
|
||||
|
||||
|
||||
## Schema
|
||||
@@ -214,16 +218,17 @@ schema:
|
||||
|
||||
#### Applicable to Elements (either Objects or Properties)
|
||||
|
||||
| Key | UX label | Required | Description |
|
||||
|--------------------------|------------------------------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| name | Name | Yes | Name of the element. |
|
||||
| physicalName | Physical Name | No | Physical name. |
|
||||
| description | Description | No | Description of the element. |
|
||||
| businessName | Business Name | No | The business name of the element. |
|
||||
| authoritativeDefinitions | Authoritative Definitions | No | List of links to sources that provide more details on the table; examples would be a link to an external definition, a training video, a GitHub repo, Collibra, or another tool. See `authoritativeDefinitions` below. |
|
||||
| quality | Quality | No | List of data quality attributes. |
|
||||
| tags | Tags | No | A list of tags that may be assigned to the elements (object or property); the tags keyword may appear at any level. Tags may be used to better categorize an element. For example, `finance`, `sensitive`, `employee_record`. |
|
||||
| customProperties | Custom Properties | No | Custom properties that are not part of the standard. |
|
||||
|
||||
| Key | UX label | Required | Description |
|
||||
|--------------------------|------------------------------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| name | Name | Yes | Name of the element. |
|
||||
| physicalName | Physical Name | No | Physical name. |
|
||||
| description | Description | No | Description of the element. |
|
||||
| businessName | Business Name | No | The business name of the element. |
|
||||
| authoritativeDefinitions | Authoritative Definitions | No | List of links to sources that provide more details on the table; examples would be a link to an external definition, a training video, a GitHub repo, Collibra, or another tool. See `authoritativeDefinitions` below. |
|
||||
| quality | Quality | No | List of data quality attributes. |
|
||||
| tags | Tags | No | A list of tags that may be assigned to the elements (object or property); the tags keyword may appear at any level. Tags may be used to better categorize an element. For example, `finance`, `sensitive`, `employee_record`. |
|
||||
| customProperties | Custom Properties | No | Custom properties that are not part of the standard. |
|
||||
|
||||
#### Applicable to Objects
|
||||
|
||||
@@ -643,6 +648,7 @@ roles:
|
||||
| roles.access | Access | No | The type of access provided by the IAM role. |
|
||||
| roles.firstLevelApprovers | 1st Level Approvers | No | The name(s) of the first-level approver(s) of the role. |
|
||||
| roles.secondLevelApprovers | 2nd Level Approvers | No | The name(s) of the second-level approver(s) of the role. |
|
||||
| roles.customProperties | Custom Properties | No | Any custom properties. |
|
||||
|
||||
|
||||
## Service-Level Agreement (SLA)
|
||||
@@ -733,7 +739,7 @@ servers:
|
||||
- **description**: A description of the server.
|
||||
- **environment**: The environment where the server operates (e.g., `prod`, `dev`, `uat`). There are no set values.
|
||||
- **roles**: An optional array of roles that have access to the server.
|
||||
- **customProperties**: Any additional custom properties specific to the server.
|
||||
- **customProperties**: Any additional custom properties specific to the server that are not part of the standard.
|
||||
|
||||
### Specific Server Properties
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
domain: seller # Domain
|
||||
dataProduct: my quantum # Data product name
|
||||
version: 1.1.0 # Version (follows semantic versioning)
|
||||
status: current
|
||||
status: active
|
||||
id: 53581432-6c55-4ba2-a65f-72344a91553a
|
||||
|
||||
# Lots of information
|
||||
@@ -10,6 +10,9 @@ description:
|
||||
purpose: Views built on top of the seller tables.
|
||||
limitations: Data based on seller perspective, no buyer information
|
||||
usage: Predict sales over time
|
||||
authoritativeDefinitions:
|
||||
type: privacy-statement
|
||||
url: https://example.com/gdpr.pdf
|
||||
tenant: ClimateQuantumInc
|
||||
|
||||
kind: DataContract
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
version: "1.0.0"
|
||||
apiVersion: "v3.0.0"
|
||||
id: "6aeafdc1-ed62-4c8f-bf0a-da1061c98cdb"
|
||||
status: "Development"
|
||||
status: active
|
||||
kind: "DataContract"
|
||||
description: {}
|
||||
schema:
|
||||
schema:
|
||||
- name: "department"
|
||||
physicalName: "department"
|
||||
physicalType: "table"
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
version: 1.0.0
|
||||
kind: DataContract
|
||||
id: 53581432-6c55-4ba2-a65f-72344a91553a
|
||||
status: current
|
||||
status: active
|
||||
name: my_table
|
||||
dataProduct: my_quantum
|
||||
apiVersion: v3.0.0
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
version: 1.0.0
|
||||
kind: DataContract
|
||||
id: 53581432-6c55-4ba2-a65f-72344a91553a
|
||||
status: current
|
||||
status: active
|
||||
name: my_quantum
|
||||
apiVersion: v3.0.0
|
||||
schema:
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
version: 1.0.0
|
||||
kind: DataContract
|
||||
id: 53581432-6c55-4ba2-a65f-72344a91553a
|
||||
status: current
|
||||
status: active
|
||||
name: my_table
|
||||
dataProduct: my_quantum
|
||||
apiVersion: v3.0.0
|
||||
|
||||
@@ -2,7 +2,7 @@ version: 1.0.0
|
||||
apiVersion: v3.0.0
|
||||
kind: DataContract
|
||||
id: 53581432-6c55-4ba2-a65f-72344a91553a
|
||||
status: current
|
||||
status: active
|
||||
name: my_table
|
||||
dataProduct: my_quantum
|
||||
schema:
|
||||
@@ -26,4 +26,3 @@ schema:
|
||||
severity: error
|
||||
rule: nullCheck
|
||||
businessImpact: operational
|
||||
|
||||
@@ -2,7 +2,7 @@ version: 1.0.0
|
||||
apiVersion: v3.0.0
|
||||
kind: DataContract
|
||||
id: 53581432-6c55-4ba2-a65f-72344a91553a
|
||||
status: current
|
||||
status: active
|
||||
name: my_table
|
||||
dataProduct: my_quantum
|
||||
schema:
|
||||
|
||||
@@ -2,7 +2,7 @@ version: 1.0.0
|
||||
apiVersion: v3.0.0
|
||||
kind: DataContract
|
||||
id: 53581432-6c55-4ba2-a65f-72344a91553a
|
||||
status: current
|
||||
status: active
|
||||
name: my_table
|
||||
dataProduct: my_quantum
|
||||
schema:
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
version: 1.0.0
|
||||
kind: DataContract
|
||||
id: 53581432-6c55-4ba2-a65f-72344a91553a
|
||||
status: current
|
||||
status: active
|
||||
name: my_table
|
||||
dataProduct: my_quantum
|
||||
schema: []
|
||||
@@ -22,4 +22,4 @@ roles:
|
||||
- role: bq_unica_user_opr
|
||||
access: write
|
||||
firstLevelApprovers: Reporting Manager
|
||||
secondLevelApprovers: 'mickey'
|
||||
secondLevelApprovers: 'mickey'
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
version: 1.0.0
|
||||
kind: DataContract
|
||||
id: 53581432-6c55-4ba2-a65f-72344a91553a
|
||||
status: current
|
||||
status: active
|
||||
name: my_quantum
|
||||
apiVersion: v3.0.0
|
||||
schema:
|
||||
@@ -99,4 +99,4 @@ schema:
|
||||
physicalType: VARCHAR(40)
|
||||
- name: zip
|
||||
logicalType: string
|
||||
physicalType: VARCHAR(15)
|
||||
physicalType: VARCHAR(15)
|
||||
|
||||
33
docs/examples/schema/kafka-schema.odcs.yaml
Normal file
33
docs/examples/schema/kafka-schema.odcs.yaml
Normal file
@@ -0,0 +1,33 @@
|
||||
apiVersion: v3.0.0
|
||||
kind: DataContract
|
||||
id: orders
|
||||
status: development
|
||||
name: Orders Event Stream
|
||||
version: 0.0.1
|
||||
schema:
|
||||
- name: Orders
|
||||
physicalName: orders
|
||||
logicalType: object
|
||||
physicalType: topic
|
||||
description: Orders Kafka topic
|
||||
properties:
|
||||
- name: cust_id
|
||||
businessName: Customer ID
|
||||
logicalType: string
|
||||
physicalType: string
|
||||
required: true
|
||||
- name: prod_id
|
||||
businessName: Product ID
|
||||
logicalType: string
|
||||
physicalType: string
|
||||
required: true
|
||||
- name: qty
|
||||
businessName: Quantity
|
||||
logicalType: integer
|
||||
physicalType: int
|
||||
required: true
|
||||
servers:
|
||||
- server: my-kafka
|
||||
type: kafka
|
||||
format: avro
|
||||
host: kafkabroker1:9092
|
||||
20
docs/examples/schema/kafka-schemaregistry.odcs.yaml
Normal file
20
docs/examples/schema/kafka-schemaregistry.odcs.yaml
Normal file
@@ -0,0 +1,20 @@
|
||||
apiVersion: v3.0.0
|
||||
kind: DataContract
|
||||
id: orders
|
||||
status: production
|
||||
name: Orders Event Stream
|
||||
version: 0.0.1
|
||||
schema:
|
||||
- name: Orders
|
||||
physicalName: orders
|
||||
logicalType: object
|
||||
physicalType: topic
|
||||
description: Orders Kafka topic
|
||||
authoritativeDefinitions:
|
||||
- url: https://schema-registry:8081
|
||||
type: implementation
|
||||
servers:
|
||||
- server: my-kafka
|
||||
type: kafka
|
||||
format: avro
|
||||
host: kafkabroker1:9092
|
||||
@@ -1,7 +1,7 @@
|
||||
version: 1.0.0
|
||||
kind: DataContract
|
||||
id: 53581432-6c55-4ba2-a65f-72344a91553b
|
||||
status: current
|
||||
status: active
|
||||
name: my_table
|
||||
dataProduct: my_quantum
|
||||
apiVersion: v3.0.0
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
version: 1.0.0
|
||||
kind: DataContract
|
||||
id: 53581432-6c55-4ba2-a65f-72344a91553c
|
||||
status: current
|
||||
status: active
|
||||
name: my_table
|
||||
dataProduct: my_quantum
|
||||
apiVersion: v3.0.0
|
||||
|
||||
@@ -2,7 +2,7 @@ version: 1.0.0
|
||||
apiVersion: v3.0.0
|
||||
kind: DataContract
|
||||
id: abc123
|
||||
status: pending
|
||||
status: in development
|
||||
servers:
|
||||
- server: my-azure
|
||||
type: azure
|
||||
|
||||
@@ -2,7 +2,7 @@ version: 1.0.0
|
||||
apiVersion: v3.0.0
|
||||
kind: DataContract
|
||||
id: abc123
|
||||
status: pending
|
||||
status: in development
|
||||
schema:
|
||||
- name: Orders
|
||||
physicalName: orders.avro.v1
|
||||
|
||||
@@ -2,7 +2,7 @@ version: 1.0.0
|
||||
apiVersion: v3.0.0
|
||||
kind: DataContract
|
||||
id: 53581432-6c55-4ba2-a65f-72344a91553a
|
||||
status: current
|
||||
status: active
|
||||
name: my_table
|
||||
dataProduct: my_quantum
|
||||
schema: []
|
||||
|
||||
@@ -2,7 +2,7 @@ version: 1.0.0
|
||||
apiVersion: v3.0.0
|
||||
kind: DataContract
|
||||
id: 53581432-6c55-4ba2-a65f-72344a91553a
|
||||
status: current
|
||||
status: active
|
||||
name: my_table
|
||||
dataProduct: my_quantum
|
||||
schema: [ ]
|
||||
|
||||
@@ -37,8 +37,10 @@
|
||||
},
|
||||
"status": {
|
||||
"type": "string",
|
||||
"description": "Current status of the dataset. Valid values are `production`, `test`, or `development`.",
|
||||
"examples": ["production", "test", "development"]
|
||||
"description": "Current status of the dataset.",
|
||||
"examples": [
|
||||
"proposed", "draft", "active", "deprecated", "retired"
|
||||
]
|
||||
},
|
||||
"servers": {
|
||||
"type": "array",
|
||||
@@ -66,6 +68,12 @@
|
||||
"limitations": {
|
||||
"type": "string",
|
||||
"description": "Limitations of the dataset."
|
||||
},
|
||||
"authoritativeDefinitions": {
|
||||
"$ref": "#/$defs/AuthoritativeDefinitions"
|
||||
},
|
||||
"customProperties": {
|
||||
"$ref": "#/$defs/CustomProperties"
|
||||
}
|
||||
}
|
||||
},
|
||||
@@ -111,6 +119,9 @@
|
||||
"$ref": "#/$defs/ServiceLevelAgreementProperty"
|
||||
}
|
||||
},
|
||||
"authoritativeDefinitions": {
|
||||
"$ref": "#/$defs/AuthoritativeDefinitions"
|
||||
},
|
||||
"customProperties": {
|
||||
"$ref": "#/$defs/CustomProperties"
|
||||
},
|
||||
@@ -2106,7 +2117,7 @@
|
||||
},
|
||||
"AuthoritativeDefinitions": {
|
||||
"type": "array",
|
||||
"description": "List of links to sources that provide more details on the dataset; examples would be a link to an external definition, a training video, a GitHub repo, Collibra, or another tool. Authoritative definitions follow the same structure in the standard.",
|
||||
"description": "List of links to sources that provide more details on the dataset; examples would be a link to an external definition, a training video, a git repo, data catalog, or another tool. Authoritative definitions follow the same structure in the standard.",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
@@ -2228,6 +2239,9 @@
|
||||
"secondLevelApprovers": {
|
||||
"type": "string",
|
||||
"description": "The name(s) of the second-level approver(s) of the role."
|
||||
},
|
||||
"customProperties": {
|
||||
"$ref": "#/$defs/CustomProperties"
|
||||
}
|
||||
},
|
||||
"required": ["role"]
|
||||
|
||||
@@ -14,6 +14,7 @@ catalogs, data quality platforms, security tools, and more.
|
||||
* [Data Contract Playground](https://data-catering.github.io/data-contract-playground/) - Playground site for creating, exporting and validating data contracts
|
||||
* [DQC.ai | DQ PLATFORM](https://www.dqc.ai/dqc-platform) - [Enhancing Data Quality with ODCS: A Standard Ensured by the DQ Platform
|
||||
](https://www.dqc.ai/post/enhancing-data-quality-with-odcs-a-standard-ensured-by-the-dq-platform)
|
||||
* [Meta Analysis](https://www.meta-analysis.fr/en/home/) - Governance repository & Data Catalog, supports ODCS via a [flexible metamodel and open API](https://www.meta-analysis.fr/en/article-en/open-data-contract-standard-adoption/)
|
||||
|
||||
## Service providers
|
||||
|
||||
|
||||
Reference in New Issue
Block a user