Connecting to the NTSPortal Elasticsearch Search API from R or Python
Connecting-to-NTSPortal-API.RmdAPI Connection, getting started
The API is called from elastic.bafg.de. Users must 1)
have been granted API access (please send a request to
ntsportal@bafg.de) and 2) must have created an API key for
themselves at ntsportal.bafg.de
(Menu->Management->Stack Management->Security->API keys,
‘control security privileges’ must be checked”)
The API documentation is found here. Detailed documentation is available for the different search languages offered, e.g. Query DSL.
ntsportal::getTableByEsql() is the simplest function for
collecting data from NTSPortal in R. It uses the ES|QL API
(See documentation)
which can be tested in the Discover page of the NTSPortal
dashboard. The data is returned as a tbl_df object (similar
to a data.frame). The limitation of ES|QL is
that nested fields (like ms2 and
compound_annotation) cannot be queried or retrieved, for
this you must use ntsportal::getTableByQuery().
ntsportal::getTableByQuery() allows fetching records
from NTSPortal as a tbl_df and allows the retrieval of
nested fields. This function uses the Query DSL language (in the form of
a list) to pass the queries. The best way to get started
with Query DSL is to use the Elasticsearch “Developer Tools” console in
the NTSPortal Dashboards.
For an example of using the API in Python see Using the Elasticsearch Search API with Query DSL for retrieving documents.
Storing of user credentials
ntsportal::connectNtsportal() stores and retrieves the
NTSPortal user credentials, it uses keyring to store the
username and password.
Under the hood
The API interface functions above use the Python package
elasticsearch to query the database (documentation). To
see how it works look at the code for
ntsportal::PythonDbComm(), which uses the
inst/pythonElasticComm/elasticSearchComm.py module to
create an elasticsearch client object in R. This client is used in many
of the interface functions found in
R/dbInterface-PythonDbComm.R.