Skip to contents

Using the NTSPortal front-end

Source: vignettes/using-ntsportal-front-end.Rmd
using-ntsportal-front-end.Rmd

Introduction

The NTSPortal UI at ntsportal.bafg.de is built using the Kibana platform (Documentation). This allows interactive viewing of the data.

Registration and user access

Access to ntsportal.bafg.de is currently restricted by IP whitelisting while it is still in development. Only static IP addresses are permitted and these must be white-listed by the BfG. Signing up for access is currently restricted to German government offices. Please contact the BfG for more information or send a request for access to ntsportal@bafg.de.

Structure

The homepage of ntsportal.bafg.de has a series of links on the left-hand side to access different dashboards (pages with different views of the data). The homepage and all linked dashboards are restricted to annotated features, meaning non-target detections for which there was a match in the Collective Spectral Library (CSL).

Example usage

Search for a compound in the database and view the spatial distribution and time series

From the homepage, following the link “Search for substances”, initially the user will see the full dataset (which the user’s role has access to). A table on the left lists all detected compounds and a map displays all detections as dots. The drop-down menus at the top of the screen allow the user to filter the data for a compound or several compounds of interest using a name or other identifiers, by the compound groups or by filtering by m/z. Once selected, the table and map will be filtered for the compound(s). By hovering on the name of the compound, a “+” button will appear, which provides “drilldowns” – links to other dashboards to view further details. For example, from here one can view the time series of the selected compound at sites where time series data is available.

Filter context in Kibana

The dashboards are views of the dataset which is available to the current user (defined through roles). Each user has an assigned role (or roles) with specific data access rights. Most dashboards initially display a large (albeit truncated) dataset and the user is expected to filter for the data for interest, e.g., by selecting specific compounds or sampling locations. This is known as the filter context and it usually applies to all visualizations within the dashboard.

Kibana provides several different filtering methods:

  • Within most dashboards drop-down menus are provided for the most important fields. This is the primary filtering method.
  • All dashboards have a search box to provide custom filters using the relatively simple KQL syntax. This requires knowledge of the field names and structure of the database.
  • To the left of the search box is a “+”-button with which custom filters can be added graphically.
  • To the right of the search box is the time selector, here the data can be filtered by time (sampling time, measurement time or import time, depending on the dashboard).

Data truncation

Depending on the visualization, the data displayed to the user may be automatically truncated to minimize latency. How much the data is truncated depends on the visualization.

Data may also need be summarized on the server before it is sent to the browser. For instance, the data for a time series of peak area of a compound in daily measurements may be too large so it is summarized to weekly data points by taking the median intensity for each week.

Truncating or summarizing is done by ElasticSearch at query time. Kibana receives the truncated data and displays it. In order to see the whole data of interest, the user is expected to narrow down the filter context.

Custom queries with Kibana Query Language

The KQL searchbox allows the user to set more complex filters within a dashboard. For a full reference on KQL see the elasticsearch documentation.

The basic form of the query is field : value, where the field names and values can be read from the tables on the left. Examples:

  • search for a compound: name: Carbamazepine
  • Search for two compounds: name: Valsartan or name: "Valsartan acid"
  • Wildcard search: station: rhein*
  • Numeric range search: mz > 237.1 and mz < 237.2

The nested fields (see field mappings) are queried with field: {field : value }, in this example the search is for all compounds with fragments 179 (lower intensity) and 194 (dominant fragment) in positive ionisation mode:

ms2: { mz > 179 and mz < 180 and int > 0.2 } and ms2: { mz > 194 and mz < 195 and int > 0.8 } and pol: pos

Notes

  1. For custom filters, once these have been set, the user must “Refresh” to send the query to the database and retrieve the new data. This is done automatically when using the drop-down menus.
  2. Some plots allow the user to “toggle-hide” datapoints (e.g. by clicking on the legend entry of a time series, the selected series is hidden). This does not fetch new data from the database. If the original query was truncated, it will remain such.