Metadata

Abstract

Every resource in DataX is described by an associated metadata JSON.

Base Metadata

DataX defines a base metadata structure to describe all the resources that it enables and provides provisions for resource-specific extensions.

An API caller typically describes the data by providing values for a few parameters in its POST call. DataX always returns back the complete MetaData object once the resource has been successfully created.

{
        "resourceType":{string},
        "resourceId":{string},
        "createTime”:{string},
        "description”:{string},
        "context" : {string},
        "links" : [
                {
                        "rel"   : "self",
                        "href" : {URI}
                },
                {
                        "rel"   : "describes",
                        "href" : {URI}
                }
        ],
        "status" : {
                "state":{string},
                "lastUpdateTime":{string},,
                "errors" : [
                        {
                                "code" : {string},
                                "message" : {string},
                                "log" : {string}
                        }
                ]
        }
                "bytes": {string},
                "extensions" : { }
}

Note

The Description field must be provided by an API caller. The Context field is optionally provided by an API caller. All the other fields are generated and returned by DataX.

JSON Property Descriptions

Property Name

Type

Description

resourceType

String

[output-only] A unique identifier for a type of resource, e.g., ‘datax- taxonomy’, ‘datax-audience’ etc. Please see resource documentation for accurate values.

resourceId

String

[output-only] A case insensitive unique ID for a resource. It will be a string of 1 to 128 characters satisfying the regular expression:

^[a-zA-Z0-9][-_.a-zA-Z0-9]{0,127}$

createTime

DateTime

[output-only] An RFC 3339 compliant string representing the date and time of the creation of this resource within DataX. For example, 2009-10- 01T18:30:02Z.

description

String

A human friendly description of this resource or its version, up to 1024 characters.

context

String

[optional] An optional identifier provided by you (the API caller) for your own reference. It can be any UTF-8 string of up to 128 characters. DataX will not try to interpret this property. It will persist this information in the meta data. This could be your reference/resource id for example.

links

Array

[output-only] A set of useful links.

links[].rel

String

[output-only] A link relation type. One of https://www.iana.org/assignments/link-relations/link-relations.xml or a URI formatted custom type with documentation at the URI. This version of DataX provides ‘Self’ (Metadata) and ‘Describes’ (Resource Data) links.

links[].href

URI

[output-only] Link of type Links[].rel

status

Object

[output-only] Current status of the Resource

status.state

String

[output-only] Current state of the resource within DataX. A resource may implement its own set of states.This property will be set to ‘ERROR’ across all resource implementations, in the event that processing of the resource fails and the resource is rejected.

status.lastUpdateTime

DateTime

[output-only] An RFC 3339 compliant string representing the date and time of the last change in status of this resource. For example, 2009-10- 01T18:30:02Z.

status.errors

Array

[output-only] A list of errors encountered while processing the resource. Please see resource specific list of Error codes and associated messages. Note, that a resource may process with errors, but still be successfully ‘accepted’. Presence of errors does not imply that the resource was rejected.

status.errors[].code

String

[output-only] An individual error code as per Appendix C.

status.errors[].message

String

[output-only] An error message as per Appendix C.

status.errors[].log

String

[output-only] Debug log generated by the component which reported this error. This is a free-form text, without any format/structure guarantees, useful only for human debugging purposes.

bytes

Number

[output-only] Size of file uploaded.

extensions

Object

[output-only] Resource specific extensions, if any.