Guide: Credential Registry

Creative Commons License
Credential Transparency Description Language (CTDL) by Credential Engine is licensed under a Creative Commons Attribution 4.0 International License.

About

This document is intended to provide an overview of the Credential Engine Registry and how to interact with it.

Status

Draft

Latest Update:

2017-04-27

Table of Contents

Top
  1. Introduction: Credential Engine Registry
    1. CTID
    2. Registry Sandbox
  2. Licenses (TBD)
    1. Open Information Assurances 1.0 (TBD)
  3. Getting Started (TBD)
    1. Setup (TBD)
    2. Signing Up (TBD)
    3. Publishing Keys (TBD)
  4. Publishing Data (TBD)
    1. JSON-LD CTDL Schema
    2. JSON Validation Schema
    3. Registry Envelope
    4. Publish API (TBD)
      1. Third-Party Proxy Publishing (TBD)
      2. Registry Proxy Publishing (TBD)
      3. Crawler-Pull Publishing (TBD)
      4. Self-Signed Custom Publishing (TBD)
  5. Extracting Data (TBD)
    1. API Overview
    2. Search API
      1. General Definition
      2. Empty Search
      3. Pagination
      4. Full Text Search
      5. Filter By Community
      6. Filter By Type
      7. Filter By Date Range
      8. Resource Specific Types
      9. Find Any Resource By Field
      10. Using Prepared Queries
    3. RDF URIs (TBD)
    4. Get Records (TBD)
  6. Credential Engine Registry Services (TBD)
    1. Search (TBD)
    2. RSA Key Generation (TBD)
  7. Credential Engine Registry Archive (TBD)
    1. Archive.org (TBD)
    2. Archive Format (TBD)
  8. References

1. Introduction: Credential Engine Registry

The Credential Engine Registry is a repository of metadata about Credentials and related entities (such as organizations, assessments, and learning opportunities) and a set of related services to enable publishing, finding, and retrieving data about these entities.

This data is described using CTDL, which allows for consistent data across a variety of sources and types of entities.

While CTDL is based on linked data principles, and individual CTDL records are intended to translate fluently to RDF documents, the Registry itself is not an RDF Triples store.

The data is organized and stored in a community based metadata registry. Using this your community can have a full data store with an api, data validation and search capabilities, by just providing a simple config with a json-schema definition.

It is used as the API engine underneath the CE/Registry and additionally comprises of the new implementation of the Learning Registry API, using a widely-used database and providing a more developer-friendly, REST-oriented environment.

1.1. CTID

Resources in the Credential Engine Registry are identified by their Credential Transparency ID (or "CTID" for short). This identifier takes the form of the text "ce-" (for "Credential Engine") prepended to a UUID, for example: ce-2199c4e3-e402-4047-817d-3eb0a783178d.

One of the goals of the CTID is to enable the resource to be resolved within the registry and reliably identified outside of it.

1.2. Registry Sandbox

Currently the Credential Engine Registry exists as a "sandbox" environment, meaning it is a semi-volatile repository intended for testing and development.

2. Licenses

2.1. Open Information Assurances 1.0

3. Getting Started

Getting Started Text

3.1. Setup

3.2. Signing Up

3.3. Publishing Keys

4. Publishing Data

Publishing Data Text

4.1 JSON-LD CTDL Schema

4.2 JSON Validation Schema

View all JSON Validation schema View Credential JSON Validation schema View Organization JSON Validation schema View Assessment JSON Validation schema View Learning Opportunity JSON Validation schema

4.3 Registry Envelope

All data the registry is dealing with has to be wrapped inside of a registry envelope. The envelope’s attributes define what kind of data the registry is processing and contain all information necessary to validate the integrity of the data.

The envelope’s structure is as follows:

{
 "envelope_type": "resource_data",
 "envelope_version": "1.0.0",
 "envelope_community": "ce_registry",
 "resource": /* JWT encoded resource from the previous step */,
 "resource_format": "json",
 "resource_encoding": "jwt",
 "resource_public_key": /* Public key in PEM format, e.g. the content from '~/.ssh/id_rsa.pem', be aware of line breaks */
}

Where:

For our example, create an 'envelope.json' with:

{
 "envelope_type": "resource_data",
 "envelope_version": "1.0.0",
 "envelope_community": "ce_registry",
 "resource": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJAY29udGV4dCI6eyJzY2hlbWEiOiJodHRwOi8vc2NoZW1hLm9yZy8iLCJkYyI6Imh0dHA6Ly9wdXJsLm9yZy9kYy9lbGVtZW50cy8xLjEvIiwiZGN0IjoiaHR0cDovL2R1YmxpbmNvcmUub3JnL2RjL3Rlcm1zLyIsInJkZiI6Imh0dHA6Ly93d3cudzMub3JnLzE5OTkvMDIvMjItcmRmLXN5bnRheC1ucyMiLCJyZGZzIjoiaHR0cDovL3d3dy53My5vcmcvMjAwMC8wMS9yZGYtc2NoZW1hIyIsImNldGVybXMiOiJodHRwOi8vcHVybC5vcmcvY3RkbC90ZXJtcy8ifSwiQHR5cGUiOiJjZXRlcm1zOkNyZWRlbnRpYWxPcmdhbml6YXRpb24iLCJAaWQiOiJ1cm46Y3RpZDplMDk1OWU5OC03OGZkLTQ5NWUtOTE4OS1lZDdkM2RhZmM3MGMiLCJjZXRlcm1zOm5hbWUiOiJTYW1wbGUgT3JnIn0.TVFa5Zjg07MDQ6XqdFYV2-9mevBA3BnzwHHj1ohXQ1eYvVRFPq63URd9x0atQ4B7vxI311gla7YW7tyT3o3AzFuLiylLlluFlXcomHYyEoWxTVlXOfaYsSdlYrA853cOoghahCS5_OxkPG5bV9jLt4nuovfbEbBZLQIcedh2g1fu2efHY7xoc3DrVQ4tNSYL4QKpOqdl3S-ccD-BP-dwx_lriwEQNZ8mI6N0BdRZDQHfksrkR4KApOoe19yQzUbS57XAnZFv5xvTowCavko4SZwgNcpcXYUhBx8UcAc6LH_0zDV-6bsjT1nv9VQV2taBfks16S99xwzbwV6IewVcWQ",
 "resource_format": "json",
 "resource_encoding": "jwt",
 "resource_public_key": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAp32A8vGxAxwgVM1pLNUb\nPH0WPB1tX6ASoyOcXvCuW0cTHGdxnbYY+3TbLmjBQSUiznUXWGO3eqTK9YU8kKAo\nApXtOZNwjBLxp5K3xZNjGv9mryqWiGN4IPQWvTR2lvLmPpNOPEhJETL9Hq9Lzzzk\nV0R/bdd2+5WxF83gV9tSH1FfmrEF5RZk8QoLCxdWxmymwF69M6AjV8KQnbZJazYK\n7dbei60Bs8Hy8OV23ehiW5kvUt7DUPBKxVtHvTySE2Ntmd/0Ib/s2bCIfZGJv2ts\nMerRRr665jRCQ43xU043qSPBLUa7TlWWiyqi5UUiWAlyPHXtxaaDJUajIYJD/1os\nCwIDAQAB\n-----END PUBLIC KEY-----\n"
}

You can check the envelope schema definition on:

4.4. Publish API

4.4.1 Third-Party Proxy Publishing

4.4.2. Registry Proxy Publishing

4.4.3. Crawler-Pull Publishing

4.4.4. Self-Signed Custom Publishing

5. Extracting Data

Extracting Data Text

5.1. API Overview

You can see and test the API here.

The main endpoint is /resources/{resource_id}. You can do here GET, POST, PUT and DELETE operations. POST, PUT and DELETE actions require an envelope. However, GET operation returns the resource without envelope. Hence, a request to http://credentialengine.org/resources/myuuid will return the resource JSON of the resource @id field is http://credentialengine.org/resources/myuuid.

Most of our endpoints have a corresponding 'info' with some extra information and links to relevant docs and specifications. For example: the endpoint has a /info, /schemas has a /schemas/info and so forth.

Below we provide a list of the 'info' endpoints and the expected response they will show

{
  metadata_communities: [ object with metadata_communities and their urls ],
  postman: 'url to postman docs',
  swagger: 'url to swagger docs',
  readme: 'url for readme',
  docs: 'url for docs folder'
}
{
  available_schemas: [ list of available schema urls ],
  specification: 'http://json-schema.org/'
}
{
    "backup_item": "ce-registry-test",
    "total_envelopes": 1024
}
{
    "POST": {
        "accepted_schemas": [ list of resource schemas for this community ]
    },
    "PUT": {
        "accepted_schemas": ["http://localhost:9292/schemas/delete_envelope"]
    }
}
{
    "PATCH": {
        "accepted_schemas": [ list of resource schemas for this community ]
    },
    "DELETE": {
        "accepted_schemas": ["http://localhost:9292/schemas/delete_envelope"]
    }
}

5.2. Search API

Search and filtering are provided on /search.

You can also use community specific endpoints, i.e: /{community-name}/search

For communities, like ce-registry which has specific resource types, you can also use endpoints like /{community-name}/{type}/search.

The search params are described below:

5.2.1. General definition

Usually takes the following format, with some modifiers which will be specified along this document:

GET /search?fts=fuzzy_search_term&filter1=term1&filter2=term2`

5.2.2. Empty Search

An empty search will perform a match_all query.

5.2.3. Pagination

You can paginate on the search results by using the page and per_page params. On the response headers we provide links for the pagination on the Link header.

For example:

# http ":9292/search?page=2&per_page=20" -h


Content-Length: 227025
Content-Type: application/json
Link: <http://localhost:9292/search?page=1&per_page=20>; rel="first", <http://localhost:9292/search?page=1&per_page=20>; rel="prev", <http://localhost:9292/search?page=12&per_page=20>; rel="last", <http://localhost:9292/search?page=3&per_page=20>; rel="next"
Per-Page: 20
Total: 223

5.2.4. Full Text Search

Try to find anything related to the provided search term. Uses the fts param:

GET /search?fts=something
GET /{community}/search?fts=something
GET /{community}/{type}/search?fts=something

5.2.5. Filter by community

there is two ways:

GET /search?community=community-name`
GET /{community-name}/search

ex: GET /ce-registry/search

5.2.6. Filter by type

by default we search for any type of data envelope, if you want only resources or paradata, use type=resource_data or type=paradata

GET /search?type=paradata

PS: notice that type is related to the envelope, i.e: which kind of data this envelope holds. This is different from the resource_type like we are going to see below.

5.2.7. Filter by date range

use the from and until filters:

GET /search?from=2016-07-20T00:00:00Z&until=2016-07-31T23:59:59Z

the date params usually follow the ISO 8601 format.

You can also provide a natural-language description for the dates. I.e:

GET /search?from=3 months ago
GET /search?until=now
GET /search?from=february 1st&until=last week

5.2.8. Resource specific types

The resource_type, refers to the resource. They are specific by community, for example: the community ce-registry has the resource_types CredentialOrganization and Credential, whilst the learning registry has no specific type.

GET /ce-registry/search?resource_type=credential
GET /ce-registry/search?resource_type=organization
GET /ce-registry/credentials/search
GET /ce-registry/organizations/search

5.2.9. Find by any resource field

You can search by any resource key. For example:

GET /ce-registry/search?ceterms:ctid=urn:ctid:9c699c33-ceb6-4e76-8009-fbfa2e443762
GET /ce-registry/search?ctid=urn:ctid:9c699c33-ceb6-4e76-8009-fbfa2e443762
# You can configure aliases for special keys on the `config.json`, i.e: ctid => ceterms.ctid

For root-level properties just follow the pattern: prop_name=value

You can also query on nested fields by providing a json piece that should be contained on the resource.

I.e., given the resource below:

decoded_resource": {
    "@type": "ceterms:Certification",
    "@context": {
        "dc": "http://purl.org/dc/elements/1.1/",
        "dct": "http://dublincore.org/terms/",
        "rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#",
        "ceterms": "http://purl.org/ceterms/terms/",
        "rdfs": "http://www.w3.org/2000/01/rdf-schema#",
        "schema": "http://schema.org/"
    },
    "ceterms:subjectWebpage": "https://www.servsafe.com/manager/food-safety-training-and-certification",
    "ceterms:name": "ServSafe Food Protection Manager Certification",
    "ceterms:purpose": [
        "Entry level within an occupation"
    ],
    "ceterms:description": "The ServSafe® program provides food safety training, exams and educational materials to foodservice managers. Students can earn the ServSafe Food Protection Manager Certification, accredited by the American National Standards Institute (ANSI)-Conference for Food Protection (CFP).",
    "ceterms:audienceLevelType": [
        "Postsecondary (Less than 1 year)",
        "Postsecondary (1-3 years)",
        "Postsecondary (3-6 years)",
        "Postsecondary (6+ years)",
        "High School"
    ],
    "ceterms:industryType": [
        {
            "@type": "ceterms:CredentialAlignmentObject",
            "ceterms:url": "https://www.census.gov/cgi-bin/sssd/naics/naicsrch?code=311&search=2012",
            "ceterms:name": "Food Manufacturing",
            "ceterms:framework": "http://www.credreg.net/naics",
            "ceterms:frameworkName": "NAICS"
        },
        {
            "@type": "ceterms:CredentialAlignmentObject",
            "ceterms:url": "https://www.census.gov/cgi-bin/sssd/naics/naicsrch?code=31141&search=2012",
            "ceterms:name": "Frozen Food Manufacturing",
            "ceterms:framework": "http://www.credreg.net/naics",
            "ceterms:frameworkName": "NAICS"
        },
        {
            "@type": "ceterms:CredentialAlignmentObject",
            "ceterms:url": "https://www.census.gov/cgi-bin/sssd/naics/naicsrch?code=72233&search=2012",
            "ceterms:name": "Mobile Food Services",
            "ceterms:framework": "http://www.credreg.net/naics",
            "ceterms:frameworkName": "NAICS"
        }
    ]
}


You can find entries that has the value "High School" on the array ceterms:audienceLevelType, using:

GET /ce-registry/search?ceterms:audienceLevelType=["High School"]

Now let's suppose you want to search for entries with an 'industryType' item with the name 'Food Manufacturing':

GET /ce-registry/search?ceterms:industryType=[{"unknown:items": [{"ceterms:name": "Food Manufacturing"}]}]
# OR
GET /ce-registry/search?ceterms:ceterms:industryType=[{"ceterms:name": "Food Manufacturing"}]

and so forth, all you need to do is provide a valid piece of json that should be contained on the resource.

5.2.10. Using prepared queries

You can add to the config.json prepared queries, i.e: query templates to be used with special keys. For example, on the config you can add the entry below:

  "prepared_queries": {
    "publisher_name": "processed_resource @> '{ \"publisher\": { \"name\": \"$term\" } }'"
  }

when you enter the following search: /community-name/search?publisher_name=Someone. It translates to the query defined above with the $term placeholder properly replaced.

RDF URIs

RDF URIs text

Get Records

Get Records text

6. Credential Engine Registry Services

Credential Engine Registry Services Text

7. Credential Engine Registry Archive

Credential Engine Registry Archive Text


REFERENCES

[Credential Engine Registry Website]
Official Credential Registry Website; http://credentialengineregistry.org/
[CTDL Metadata Viewer]
Metadata viewer for CTDL; http://credreg.net/ctdl/terms/