datasources

(TODO: Document how the datasources system works and how it can be configured. Show the steps for a simple datasource integration.)

The idea is to allow any REST or SPARQL API to be added to a Conzept system and render topics in the sidebar.

Features:

  • Make adding a new datasource as easy as possible, but still flexible in terms of the rendered output.
  • Make the API search results show up in the same style and results paging mechanics, as the other datasources.
  • Let the topic results be augmented from Wikidata, if a Qid for the topic is provided.
  • Have auto-completion work too (if wanted and possible).
  • Allow the user to choose which datasources to use.

use cases

  • Allow for adding extra “data sources” to a Conzept system, so other concepts - beyond Wikipedia and Wikidata - can also be presented.
  • Any organization that wants to use the Conzept user-interface for showing and augmenting their own knowledge base.

status

datasources options for the user
  • The datasources system is now somewhat working (configuration example: src/data/datasources.js), with these remaining issues:
    • BUG: There is a paging logic bug in the current code, which needs to be found and fixed.
    • BUG: Indicate when there are “no results found” for a search, or when a datasource is “unreachable”
    • Cleanup the access function API (see the “technical roadmap” notes below)
  • Technical roadmap:
    • Research how to further generalize the datasource integration, using the common result-format used on Conzept (based on the Wikipedia API result structure)
    • Add hooks for pre- and post-processing of the API data.
    • Add hooks to enrich the API data (with Wikipedia and Wikidata information)
    • Test the datasource system on other real datasources (Sparql and REST).
    • Allow for datasource activation via an URL parameter (so these searches can be linked to) and store these in a cookie.

adding a new datasource

  1. Add the new datasource to the datasource-configuration file here: src/data/datasources.js
  2. Add a new datasource directory (using the name of the datasource, eg. “gleif”)
  3. Add a new JS file in that directory, implementing the required API access functions, example: src/datasources/gleif/index.js