Integrate Data Source in 60 Minutes!

Last updated: March 16, 2021

Reading time: 14 min

Tags: V1 APIs, Data Source, Harmonized Data

Follow this step by step procedure to Integrate a Data Source system in Platform of Trust Sandbox and start Harmonized Data Flow in the Platform in less than an hour.

This tutorial is based on a sample connector implemented by Platform of Trust

PREREQUISITE: You should already have a connector hosted to send requests to the desired Data source and fetch responses from it.

Note: Keep in mind, Version 1.0 of Platform of Trust APIs has been used in this demonstration.

Step 1: Register to Platform of Trust Sandbox

Learn more about Sandbox capabilities from our Sandbox developer guide.

Step 2: Get Bearer Token

Get your Bearer token to start making calls to Platform of Trust APIs in Sandbox.

NOTE: Bearer token expires in 24 hours. So Make sure to get the regenerated one to continue.

Step 3: Create a Group

The first step of registering any Data Product or Application in the Platform Sandbox is to create a Group identity.

You can also use the World app to create a Group .

Or you can use the following cURL requests:

            

    
    curl -i --request POST \
  --url https://api-sandbox.oftrust.net/identities/v1 \
  --header 'authorization: Bearer eyJ0eXAiOiJKV1QiL...RtaWrVlxDG2awTlixI-fnVsfjbkJ1uGXVUnVU' \
  --header 'content-type: application/json' \
  --data '{
  "@context": "https://standards.oftrust.net/v2/Context/Identity/Group/",
  "@type": "Group",
  "data": {
    "name": "demo-group"
  }
}'


          

On success, you'll receive a similar response:

            

    
    {
   "@context":"https://standards.oftrust.net/v2/Context/Identity/Group/",,
   "@type":"Group",
   "@id":"0db1e898-2184-4eb5-9b9b-4d259c5cb4de",
   "data":{
      "name":"demo-group"
   },
   "metadata":{
      "createdAt":"2020-05-06T09:52:34+00:00",
      "createdBy":"01725951-3bd4-4c6f-875c-df1d07d7120f",
      "updatedAt":"2020-05-06T09:52:34+00:00",
      "updatedBy":"01725951-3bd4-4c6f-875c-df1d07d7120f"
   },
   "inLinks":[

   ],
   "outLinks":[

   ]
}


          

Make sure to save the group @id and createdBy values to use further.

NOTE: In Platform of Trust, every user is an identity, hence it has an identity ID. createdBy is your user's identity ID.

Lastly, to register an application or create a data product, you need to be memberOf in a group. Execute in the same terminal the following:

            

    
    curl -i --request POST \
  --url https://api-sandbox.oftrust.net/identities/v1/{createdBy_id}/link/{group_id} \
  --header 'authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGc...RtaWrVlxDG2awTlixI-fnVsfjbkJ1uGXVUnVU' \
  --header 'content-type: application/json' \
  --data '{
  "@context": "https://standards.oftrust.net/v2/Context/Link/Role/MemberOf/",
  "@type": "MemberOf",
  "data": {}
}'


          

In successful response, you'll receive the link @id from you user to the created group:

            

    
    HTTP/1.1 201 Created
date: Wed, 06 May 2020 10:05:01 GMT
server: uvicorn
content-type: application/json
transfer-encoding: chunked
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload;

{
   "@context":"https://standards.oftrust.net/v2/Context/Link/Role/MemberOf/",
   "@type":"MemberOf",
   "@id":"67e1dfba-7092-43d3-a2bf-54eaa4cba0b1",
   "from":"01725951-3bd4-4c6f-875c-df1d07d7120f",
   "to":"0db1e898-2184-4eb5-9b9b-4d259c5cb4de",
   "data":{

   },
   "metadata":{
      "createdAt":"2020-05-06T10:05:01+00:00",
      "createdBy":"01725951-3bd4-4c6f-875c-df1d07d7120f",
      "updatedAt":"2020-05-06T10:05:01+00:00",
      "updatedBy":"01725951-3bd4-4c6f-875c-df1d07d7120f"
   }
}


          

Now you can use this group to register Application or create Data Products.

NOTE: You can try out our Insomnia REST API workspace to access and test Platform of Trust APIs in Sandbox.

Read the Insomnia Workspace Guide

Step 4: Create Identity Network

It is always the best practice to draw a demo diagram of the identities you want to create and link.

Quick Start Guide - Identity Network Dark (1).jpg

4.1 Create an Identity

To create an identity :

            

    
    curl -i --request POST \
  --url https://api-sandbox.oftrust.net/identities/v1 \
  --header 'authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9eyJzY29w...DVs5aaf' \
  --header 'content-type: application/json' \
  --data '{
  "@context": "https://standards.oftrust.net/v2/Context/Identity/Structure/Building/",
  "@type": "Building",
  "data": {
    "name": "Glass building"
  }
}'


          

Here, we have created an ìdentityof typeBuilding`.

Make sure you correctly define the context and type of the identity you want to create. You can find out more about contexts at Platform of Trust Ontology Viewer .

On success, you will receive the id of the created identity. Make sure to save it for creating links later.

            

    
    HTTP/1.0 201 Created

{
  "@context": "https://standards.oftrust.net/v2/Context/Identity/Structure/Building/",
  "@type": "Building",
  "@id": "ec2d273b-b866-496e-b682-79b061dbf71f",
  "inLinks": [],
  "outLinks": [],
  "data": {
    "name": "Glass building"
  },
  "metadata": {
    "createdAt": "2019-09-11T11:24:23+00:00",
    "createdBy": "33237067-e72c-4f26-b78b-9f9e234b2e7d",
    "updatedAt": "2019-09-11T11:24:23+00:00",
    "updatedBy": "33237067-e72c-4f26-b78b-9f9e234b2e7d"
  }
}


          

Make sure to save the @id of the identity created as they are required to create links.

4.2 Link two Identities

To create a link between two existing identities:

            
    curl --request POST \
  --url https://api-sandbox.oftrust.net/identities/v1/<INSERT_FROM_IDENTITY_ID>/link/<INSERT_TO_IDENTITY_ID> \
  --header 'authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9eyJzY29w...DVs5aa' \
  --header 'content-type: application/json' \
  --data '{
  "@context": "https://standards.oftrust.net/v2/Context/Link/BelongsTo/",
  "@type": "BelongsTo"
}'

You need to carefully select id of the FROM_IDENTITY and TO_IDENTITY in the cURL request. So it is better to define the link directions in the diagram.
You also need to correctly define the context and type of the link.
Read more about links in Platform of Trust Ontology Viewer

On success, you will receive id of the created link. Make sure to save it for further use.

            

    
    HTTP/1.0 201 Created

{
  "@context": "https://standards.oftrust.net/v2/Context/Link/BelongsTo/",
  "@type": "BelongsTo",
  "@id": "be7a2c57-03d8-46f4-aaf0-2b1ca118ef5c",
  "from": "8ac7494b-b7bc-4a63-a253-4b9b1887b262",
  "to": "a6b5a74e-bd98-4c9b-9561-932877258833",
  "data": {},
  "metadata": {
    "createdAt": "2019-09-12T09:49:24+00:00",
    "createdBy": "33237067-e72c-4f26-b78b-9f9e234b2e7d",
    "updatedAt": "2019-09-12T09:49:24+00:00",
    "updatedBy": "33237067-e72c-4f26-b78b-9f9e234b2e7d"
  }
}


          

NOTE: it is important that first, you create two adjacent identities and then link them. Then create the 3rd identity and link it. Continue this process until you have created your desired identity network.

Step 5: Register an Application

To consume the harmonized data flow from the Platform, you need to register an Application in the Sandbox.

You can register an application in Sandbox by using Application API .

The request format is:

            

    
    curl --request POST \
  --url https://api-sandbox.oftrust.net/apps/v1 \
  --header 'Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzY29w...DVs5aaf' \
  --header 'Content-type: application/json' \
  --data '{
    "name": "Example Application",
    "description": "Application description",
    "privacyPolicyUrl": "http://example.com/privacy.html",
    "webPageUrl": "http://example.com/application.html",
    "iconUrl": "http://example.com/icon.png",
    "scopes": "",
    "defaultScopes": "",
    "redirectUris": "https://example.com/auth-callback",
    "defaultRedirectUri": "https://example.com/auth-callback",
    "groupId": "{group_id}"
  }'


          

You only need to provide name and groupID in the cURL request. groupID is of the Group created in step 3.

On success, you will receive a response similar to this:

            

    
    {
  "@context": "https://standards.oftrust.net/v2/Context/Identity/Digital/Software/App/",
  "@type": "App",
  "@id": "8c40d6d4-2dbb-4f8f-987f-5c68e7475d79",
  "data": {
    "iconUrl": "http://example.com/icon.png",
    "privacyPolicyUrl": "http://example.com/privacy.html",
    "description": "Application description",
    "webPageUrl": "http://example.com/application.html",
    "name": "Example Application"
  },
  "metadata": {
    "createdAt": "2020-02-14T07:18:22+00:00",
    "createdBy": "e224629d-3f4f-4cb2-8823-fdc839e5da98",
    "updatedAt": "2020-02-14T07:18:22+00:00",
    "updatedBy": "e224629d-3f4f-4cb2-8823-fdc839e5da98"
  },
  "inLinks": [],
  "outLinks": [],
  "client": {
    "clientId": "8c40d6d4-2dbb-4f8f-987f-5c68e7475d79",
    "redirectUris": "https://example.com/auth-callback",
    "defaultRedirectUri": "https://example.com/auth-callback",
    "scopes": "",
    "defaultScopes": "",
    "clientSecrets": [
      "4RbY09PR16F74VacbFhIsxzAJTytJfplpgZRdqE_tp4"
    ]
  },
  "tokens": [
    {
      "access_token": "wvkBrsvjBEigwd-PA...1h5vweXnzCEvpkapESHz6sxE",
      "expires_in": 63072000,
      "token_type": "Bearer"
    },
    {
      "access_token": "CkT5COQfHtbuih1a2...1h5vweXnzCEvpkapESHz6sxE",
      "expires_in": 63072000,
      "token_type": "Bearer"
    }
  ]
}


          

NOTE: Make sure to save clientId, clientSecrets, and the two access_tokens for future use. They will be required for data consumption.

You can also Register an Application in Sandbox using the World app.

You can find out more about Building Applications in the platform in relevant resources.

Step 6: Register Data Product

Data Product is used to fetch harmonized data related to your created identity network.

To register a Data Product in the Platform:

6.1 Create a Connector

You need to have a connector to make calls to the Data Source API in order to fetch harmonized data.
Below is a connector skeleton, written in python to get you started.
Make sure to carefully define the parameters that would be accepted by the connector.

            

    
    import base64
import json
import settings
from jwt.algorithms import RSAAlgorithm

# List of supported headers, and if they're required or not.
SUPPORTED_HEADERS = {
    'X-Pot-Signature': {
        'required': True
    },
    'X-App-Token': {
        'required': True
    },
    'X-User-Token': {
        'required': False
    }
}

# Get the request headers from the request object.
headers = dict()
# Get the body from the request json body.
body = dict()

# Keep the Platform of Trust public key e.g. in the settings.
public_pem = settings.POT_PUBLIC_KEY

# Loop through the request headers and validate them.
for header, rules in SUPPORTED_HEADERS.items():
    header = header.lower()
    if rules['required'] and header not in headers:
        raise Exception(
            f'Missing required header "{header}"'
        )

# Convert the request body to a string, sorting the keys, without 
# indentation and using separators comma (,) and colon (:)
payload_string = json.dumps(
    body,
    sort_keys=True,
    indent=None,
    separators=(',', ': ')
).strip()

alg_obj = RSAAlgorithm(RSAAlgorithm.SHA256)  # RSA SHA256 signature
key = alg_obj.prepare_key(public_pem)
# Verify the payload and signature against the Platform of Trust public 
# key
is_valid = alg_obj.verify(
    payload_string.encode('utf-8'), 
    key, 
    base64.b64decode(headers['X-Pot-Signature'].encode('utf-8'))
)
if not is_valid:
    raise Exception('Signature validation failed')


          

Check out Setup A Connector guide for more details.

You can also use the Connector Engine service of Platform of Trust to implement and customize your connector. Contact us with the subject "Interested in Building Connectors".

6.2 Create a Data Product

You can create a Data Product in Sandbox with Product API

            

    
    curl -i --request POST \
  --url https://api-sandbox.oftrust.net/products/v1 \
  --header 'authorization: Bearer eyJ0eXAiOi...fnVsfjbkJ1uGXVUnVU' \
  --header 'content-type: application/json' \
  --data '{
  "dataContext": "https://standards.oftrust.net/v2/Context/DataProductOutput/?v=2.0",
  "parameterContext": "https://standards.oftrust.net/v2/Context/DataProductParameters/?v=2.0",
  "productCode": "example-product-60",
  "name": "60 Minute Data Product",
  "groupId": "0db1e898-2184-4eb5-9b9b-4d259c5cb4de",
  "translatorUrl": "https://api-external-sandbox.oftrust.net/lh-translator/v1/fetch",
  "organizationPublicKeys": [
      {
        "url": "https://api-external-sandbox.oftrust.net/lh-translator/v1/public.key",
        "type": "RsaSignature2018"
      }
  ]
}'


          

NOTE: productCode needs to be unique!

keep in mind to correctly define the dataContext and parameterContext for your Data Product.
groupID is of the Group created in step 3

NOTE: translatorUrl is the server URL where you are hosting the implemented connector in step 6.1

organizationPublicKeys is a RsaSignature2018 type public key for the hosted connector.

On success, you will receive a response similar to the following:

            

    
    HTTP/1.0 201 Created

{
  "@context": "https://standards.oftrust.net/v2/Context/Identity/Product/DataProduct/",
  "@type": "DataProduct",
  "@id": "https://api-sandbox.oftrust.net/products/v1/example-product-1",
  "productCode": "example-product-60",
  "dataContext": "https://standards.oftrust.net/v2/Context/DataProductOutput/?v=2.0",
  "parameterContext": "https://standards.oftrust.net/v2/Context/DataProductParameters/?v=2.0",
  "translatorUrl": "https://example.com/translator/url",
  "name": "60 Minute Data Product",
  "organizationPublicKeys": [
    {
      "type": "RsaSignature2018",
      "url": "https://example.com/example.pub"
    }
  ],
  "description": null,
  "imageUrl": null,
  "identityId": "5d58025b-3764-4f3d-8d09-5e17b064f0f9"
}


          

NOTE: Make sure to save the identityId for the created Data Product as it would be needed later connect it to a Sensor.

Read more about Data Product in Data Product 101 guide.

Step 7: Connect Data Product to a Sensor

To connect a Data Product to a Sensor:

            

    
    curl --request POST \
  --url https://api-sandbox.oftrust.net/identities/v1/{sensorId}/link/{dataProductId} \
  --header 'authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9eyJzY29w...DVs5aa' \
  --header 'content-type: application/json' \
  --data '{
  "@context": "https://standards.oftrust.net/v2/Context/Link/AtDataProduct/",
  "@type": "AtDataProduct",
  "data": {
      "ids": {
          "site_name": "some_site_name"
      },
      "capabilities": ["TEMPERATURE"]
  }
}'


          

NOTE: In the cURL request, use @ids of the Sensor and Data Product of your choice.

Here, the link context has to be of type AtDataProduct and the ´type´ be AtDataProduct

The link direction has to be from the Data Product to the Sensor.

NOTE: You need to include the parameters that would be received by your connector. For this demo, the parameters defined are:

"data": {
"ids": {
"site_name": "some_site_name"
},
"capabilities": ["TEMPERATURE"]
}

On success, you will receive a response similar to this:

            

    
    HTTP/1.0 201 Created

{
  "@context": "https://standards.oftrust.net/v2/Context/Link/AtDataProduct/",
  "@type": "AtDataProduct",
  "@id": "be7a2c57-03d8-46f4-aaf0-2b1ca118ef5c",
  "from": "5d58025b-3764-4f3d-8d09-5e17b064f0f",
  "to": "a6b5a74e-bd98-4c9b-9561-932877258833",
  "data": {
      {
          "ids": {
              "site_name": "some_site_name"
          },
          "capabilities": ["TEMPERATURE"]
      }
  },
  "metadata": {
    "createdAt": "2019-09-12T09:49:24+00:00",
    "createdBy": "33237067-e72c-4f26-b78b-9f9e234b2e7d",
    "updatedAt": "2019-09-12T09:49:24+00:00",
    "updatedBy": "33237067-e72c-4f26-b78b-9f9e234b2e7d"
  }
}


          

Step 8: Test your Data Product

The final step before starting the harmonized data flow in the Platform.

Make a request to the Broker API to test your Data Product. You also need to know how to construct a valid request

NOTE: Broker API requires a header called x-pot-signature, which must be used in the API request within the 5 seconds of its generation. This is to ensure same signature is not used in subsequent API requests.

To generate an x-pot-signature and send a request to Broker API at the same time, customize the following python script:

            

    
    import json
import hmac
import hashlib
import base64
import http.client
from datetime import datetime

# App secret and token
secret = "<APP_SECRET_GENERATED_WITH_APPLICATION_REGISTRATION>"
app_token = "<APP_TOKEN_GENERATED_WITH_APPLICATION_REGISTRATION>"
timestamp = datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%S.000Z")
body = {
    "timestamp": timestamp,
    "productCode": "<DATA_PRODUCT_CODE>",
    "parameters": {
        "ids": [
            {"id": "<some_id>"}
        ],
        "startTime": "2020-02-10 13:06:40",
        "endTime": "2020-02-10 21:20:54",
        "dataTypes": ["TEMPERATURE"]
    },
}
body_hash = json.dumps(
    body, sort_keys=True, indent=None, separators=(",", ": ")
).strip()
digest = hmac.new(
    secret.encode("utf-8"), body_hash.encode("utf-8"), hashlib.sha256
).digest()
signature = base64.b64encode(digest).decode()
conn = http.client.HTTPSConnection("api-sandbox.oftrust.net")
headers = {
    "content-type": "application/json",
    "x-pot-signature": signature,
    "x-app-token": app_token,
}
conn.request("POST", "/broker/v1/fetch-data-product", json.dumps(body), headers)
res = conn.getresponse()
data = res.read()
print(data.decode("utf-8"))


          

Here, x-app-token is the access_token you obtained while registering an Application in step 5. secret refers to the clientSecrets from that application.

timestamp refers to the time when the request was sent. This needs to be in RFC3339 format.

productCode is from the Data Product you created in step 6.2.

parameters are additional parameters that the connector needs. Different products and connectors require different parameters. You should check the parameters list from Data Provider's translator.

x-pot-signature is A HMAC-SHA256 signature in base64 encoded format. The signature is created from the request payload and clientSecrets obtained while registering an Application in step 5.

NOTE: Make sure to add in the request payload the correct parameters in the that can be accepted and processed by the connector you created in step 6.1

Save the script. Execute the python script in your terminal with the following command:

python script_name.py

On success, depending on the connector accepting the parameters and returning a response from the data source, you should see a response similar to the following:

            

    
    {
   "@context":"https://standards.oftrust.net/v2/Context/DataProductOutput/?v=2.0",
   "data":{
      "@context":"https://standards.oftrust.net/v2/Context/Identity/Product/DataProduct/",
      "@type":"DataProduct",
      "items":[
         {
            "id":{
               "id":"Valtuustotalo_kokoushuone4-temp"
            },
            "data":[
               {
                  "timestamp":"2020-02-10 13:30:00",
                  "value":22.0,
                  "type":"Temperature",
                  "unit":"\u00b0C"
               },
               {
                  "timestamp":"2020-02-10 14:00:00",
                  "value":21.0,
                  "type":"Temperature",
                  "unit":"\u00b0C"
               },
               {
                  "timestamp":"2020-02-10 21:00:00",
                  "value":22.0,
                  "type":"Temperature",
                  "unit":"\u00b0C"
               }
            ]
         }
      ]
   },
   "signature":{
      "type":"RsaSignature2018",
      "created":"2020-04-22T10:28:04+00:00",
      "creator":"https://api-external-sandbox.oftrust.net/dummy-translator/v1/public.key",
      "signatureValue":"zR35LeCcwZ94...XS0kzeM58CEUCTpMPrM/N89Jo6hQhhogr2Z8QfnXLY6uOgi7fg=="
   }
}


          

This ensures the created connector is functioning properly and is capable to fetch data from the connected data source and pass it to Broker API in harmonized format.

Ready to get started?

Have a wishlist?

Send Us Feedback