Tags: V1 APIs
, Connector
, Harmonized Data
As a data provider who wants to integrate their data sources with Platform of Trust, you have to learn about connectors and how to set up your own.
A component which connects the raw data source to the data product and translates the data model to standard format.
Connector
Read more about Connector
in Data Product 101 guide
NOTE: The connector MUST validate the signature coming from the Platform of Trust Data Broker, as well as the headers.
Note: Keep in mind that, V1 APIs of Platform of Trust have been used in this demonstration.
NOTE: If you're using a framework, such as Bottle, the headers might work using whichever case you want, but if you're not using a framework, you need to lowercase the header name in the request, e.g. `'X-Pot-Signature'.lower()
` to make the validation pass.
import base64
import json
import settings
from jwt.algorithms import RSAAlgorithm
# List of supported headers, and if they're required or not.
SUPPORTED_HEADERS = {
'X-Pot-Signature': {
'required': True
},
'X-App-Token': {
'required': True
},
'X-User-Token': {
'required': False
}
}
# Get the request headers from the request object.
headers = dict()
# Get the body from the request json body.
body = dict()
# Keep the Platform of Trust public key e.g. in the settings.
public_pem = settings.POT_PUBLIC_KEY
# Loop through the request headers and validate them.
for header, rules in SUPPORTED_HEADERS.items():
header = header.lower()
if rules['required'] and header not in headers:
raise Exception(
f'Missing required header "{header}"'
)
# Convert the request body to a string, sorting the keys, without
# indentation and using separators comma (,) and colon (:)
payload_string = json.dumps(
body,
sort_keys=True,
indent=None,
separators=(',', ': ')
).strip()
alg_obj = RSAAlgorithm(RSAAlgorithm.SHA256) # RSA SHA256 signature
key = alg_obj.prepare_key(public_pem)
# Verify the payload and signature against the Platform of Trust public
# key
is_valid = alg_obj.verify(
payload_string.encode('utf-8'),
key,
base64.b64decode(headers['X-Pot-Signature'].encode('utf-8'))
)
if not is_valid:
raise Exception('Signature validation failed')
Two contexts MUST be defined:
The parameter context defines the parameters that are passed to the connector. In the below example is a JSON payload that is sent to the Platform of Trust Data Broker. The parameter context MUST define what the keys in the parameters
object means, and what their type is.
{
"@context": "https://standards.oftrust.net/v2/Context/DataProductParameters/Sensor/?v=2.0",
"timestamp": "2020-02-10T14:36:25+00:00",
"productCode": "unique-product-code",
"parameters": {
"ids": [
{"id": "apartament1"}
],
"dataTypes": ["MeasureAirTemperatureCelsiusDegree"]
}
}
The data context MUST define the actual data response that the connector returns.
In the below example the connector returns the actual data in the data
object. The data context is the @context
within the object. The data context SHOULD define the keys within the data object, e.g. apartmentId
, name
, rooms
, balcony
etc.
{
"@context": "https://standards.oftrust.net/v2/Context/DataProductOutput/Sensor/?v=2.0",
"data": {
"sensors": [
{
"id": {"id": "apartament1"},
"measurements": [
{
"@type": "MeasureAirTemperatureCelsiusDegree",
"timestamp": "2020-02-10T13:29:03+00:00",
"value": 26.0,
},
{
"@type": "MeasureAirTemperatureCelsiusDegree",
"timestamp": "2020-02-10T13:59:38+00:00",
"value": 25.0,
},
{
"@type": "MeasureAirTemperatureCelsiusDegree",
"timestamp": "2020-02-10T14:30:11+00:00",
"value": 26.0,
},
],
},
]
},
"signature": {
"created": "2020-02-10T14:36:40+00:00",
"creator": "https://example.com/public-key",
"signatureValue": "eyXDfjefphhfwe...gwrg+wgr0==",
"type": "RsaSignature2018",
},
}
The signature information MUST include the signature type, RsaSignature2018
, the created timestamp when the data was created, who the creator is (the public key URL used to validate the signature) and the signature value.
The signature is generated by adding a key __signed__
in the data object and using the corresponding private key to generate the signature.
Python example of how to sign the data:
import base64
import json
import settings
from copy import copy
from datetime import datetime, timezone
from jwt.algorithms import RSAAlgorithm
# This is the private key matching the `creator` public key.
private_pem = settings.PRIVATE_KEY
# Get the created_at timestamp in RFC3339 format
created_at = str(datetime.now(timezone.utc).isoformat(timespec='seconds'))
# The data contains the actual data to return to the application.
data = {
"@context": "https://standards.oftrust.net/v2/Context/Identity/Space/Apartment/",
"apartmentId": 123456,
"name": "A 18",
"rooms": "4",
"balcony": True
}
# Make a copy of the data, and add the __signed__ key to it
sign_data = copy(data)
sign_data['__signed__'] = created_at
# Create a string of the data, sort the keys, without indentation and
# separators comma (,) and colon (:)
payload_hash = json.dumps(
sign_data,
sort_keys=True,
indent=None,
separators=(',', ': ')
).strip()
alg_obj = RSAAlgorithm(RSAAlgorithm.SHA256) # RSA SHA256 signature
key = alg_obj.prepare_key(private_pem)
signature = alg_obj.sign(payload_hash.encode('utf-8'), key)
# Base64 encode the signature value and use it in the response
# `signatureValue`
signature_value = base64.b64encode(signature).decode("utf-8")
When the connector is implemented, using any of the below-mentioned methods, you need to register the data product in Platform of Trust, see the Data Product Guide
The first option is to build a connector yourself by forking the code of one of the connectors from our GitHub repository. For example, we have Entsoe connector to get electricity prices from the Entsoe API. Almost all connectors include with automated tests in the Robot framework. Check out the GitHub repository to get the instructions to run the connector in your local workstation:
The easiest way to start developing your connector is to start from the multi-connector component from our GitHub repo. This connector is currently actively developed and features are added to it periodically.
This connector provides a huge starting point for those who want to develop connectors. Basically, building a connector, in most cases, only involves writing a few configuration files.
Another benefit of using this is that it already comes in docker format so it's really easy to get is started.
For more details on how this works please read the repository README - this explains, in detail, the steps that you need to take in order to set up a connector using this architecture.
The second option is the use of an integration platform. We have a list of dedicated partners who are willing to implement customized connectors tailored to your needs. Contact us for more information about this with the message "Require an integration platform".
The open sandbox is your friend! It's an isolated environment for testing applications and data product integrations. Read more from Sandbox Guide.