10 Enabling modules

This section describes how to enable various Oskari functionalities or modules.

10.1 How to enable thematic maps

Oskari can display statistical data and link it to map layers creating thematic maps. Currently choropleth maps are supported for areal data. The statistical data are combined with the map layer data with an ID field that must match between the data. Supported data sources for statistical data are PX-Web, SDMX REST and Sotkanet REST, more support can be added via adapters written in Java. When the requested statistical data is multi-dimensional the user has to slice each dimension to reach a tabular form of data.

See requirements for using the thematic maps functionality in Oskari application.

See instructions for configuring the thematic maps functionality.

10.1.1 Requirements for thematic maps

Note! This functionality uses Redis heavily for caching so it needs to be available for the server to work properly.

Frontend

These are actions that enable the functionality to work on your apps.

Vector features support

The VectorLayerPlugin provides support for the adding vector features to the map and is required for thematic maps to work properly. The map visualization is done by adding regions for a region set as vector features to the map. The plugin needs to be started with mapfull bundles config referencing it in the plugins array:

{ "id" : "Oskari.mapframework.mapmodule.VectorLayerPlugin" }

The code for VectorLayerPlugin is included in the mapmodule and the sample application has it in the configuration so you shouldn't really need to do anything about it but if you don't see any regions on the map this can be the cause.

If you for some reason don't have you should do an application specific Flyway-migration to add the plugin to the appsetups that you need it in. See server section for details.

User-interface for thematic maps

For bundling in the user interface you will need to include the statsgrid bundle in your applications main.js.

If you always show it for users you should use this to include it on the application:

import 'oskari-loader!oskari-frontend/bundles/statistics/statsgrid/bundle.js';

OR if you want to show it to only some users you can reduce the amount of code all users need to download by using the "lazy loader". For example on the embedded map application you want to use this since most of the embedded maps don't usually include thematic maps:

import 'oskari-lazy-loader?statsgrid!oskari-frontend/bundles/statistics/statsgrid/bundle.js';

After modifying the main.js you will need to run npm run build to update the build artifacts in the dist/[version] folder.

After building a new version with the statsgrid bundle included you can test it out in the browser by running this on the developer console:

Oskari.app.playBundle({ bundlename : 'statsgrid' });

You should see the user-interface for the functionality show up on the geoportal page.

Server

Adding the functionality to geoportal user interface

After bundling in the functionality on the frontend build you will need to tell the server to start the functionality in the appsetups you want to use it with. Usually this is all appsetups of type USER and DEFAULT.

The actual thematic maps user interface is provided by another bundle: statsgrid that needs to be added to the geoportal views.

You can use a Flyway-migration to add it in your own server-extension:

package flyway.[your module];

import org.flywaydb.core.api.migration.BaseJavaMigration;
import org.flywaydb.core.api.migration.Context;
import org.oskari.helpers.AppSetupHelper;

/**
 * Adds statsgrid info bundle to all default/user appsetups
 */
public class V1_0_0__add_statsgrid_bundle extends BaseJavaMigration {

    public void migrate(Context context) throws Exception {
        AppSetupHelper.addBundleToApps(context.getConnection(), "statsgrid");
    }
}

Note that you need to change the package to match your applications module (in the sample-server-extension the module is called example and it's located here: https://github.com/oskariorg/sample-server-extension/tree/master/app-resources/src/main/java/flyway/example). You will also need to change the version number (in the class name) to match your application. Picking the next number from the current version is ok. If this is your first migration for the app you will also need to define the module to be activated in oskari-ext.properties:

db.additional.modules=myplaces, userlayer, [your module]

If you want to test is out before doing the Flyway-migration on the server side you can also add it to a single appsetup by running the below SQL. We recommend using Flyway-migration as you will most likely want to add it to any appsetups the user might have saved in addition to the default appsetup. Replace [appsetup_id] in the SQL below with the id of your default appsetup:

INSERT INTO oskari_appsetup_bundles (appsetup_id, seqno, bundle_id, config, state)
       VALUES ([appsetup_id],
        (SELECT (max(seqno) + 1) FROM oskari_appsetup_bundles WHERE appsetup_id = [appsetup_id]),
        (SELECT id FROM oskari_bundle WHERE name = 'statsgrid'),
        (SELECT config FROM oskari_bundle WHERE name = 'statsgrid'),
        (SELECT state FROM oskari_bundle WHERE name = 'statsgrid'));

Note! You don't need to add the bundle to publish-template or any embedded maps. If the user decides to publish a thematic map, the bundle is automatically added to the embedded maps bundles.

oskari-server/Maven dependencies

The server-side code for thematic maps are included by default on the sample application, but if you have an oskari-server-extension (like you propably should) make sure you have the dependencies included in the webapp-map/pom.xml:

<dependency>
    <groupId>org.oskari</groupId>
    <artifactId>control-statistics</artifactId>
    <version>${oskari.version}</version>
</dependency>
<!-- Statistics plugins -->
<dependency>
    <groupId>org.oskari</groupId>
    <artifactId>service-statistics-pxweb</artifactId>
    <version>${oskari.version}</version>
</dependency>
<dependency>
    <groupId>org.oskari</groupId>
    <artifactId>service-statistics-unsd</artifactId>
    <version>${oskari.version}</version>
</dependency>
<!-- /Statistics plugins -->

Take a look at the sample webapp-map for details: https://github.com/oskariorg/sample-server-extension/blob/1.4.1/webapp-map/pom.xml#L56-L69. There might be additional adapters available also. Take a look at the folders under oskari-server starting with 'service-statistics-'.

10.1.2 Configuring thematic maps functionality

See requirements for enabling the code that powers the thematic maps functionality.

Adding regionsets as maplayers

Regionsets used for thematic maps are configured in pretty much the same way you might register a WFS-layer for Oskari database:

INSERT INTO oskari_maplayer(type, url,
                    name, dataprovider_id,
                    locale,
                    attributes, internal, srs_name)
VALUES(
    'statslayer', 'http://mydomain.com/geoserver/wfs',
    'mylayer', (SELECT MAX(id) FROM oskari_dataprovider),
    '{ "en" : {
        "name":"Municipalities"
    }}',
    '{
        "statistics" : {
            "regionIdTag":"id",
            "nameIdTag":"name"
        }
    }', true, 'EPSG:4326');

Where

• type of the layer is 'statslayer'

• the url and name should match a layer in a WFS-service

• locale is a JSON object with language code at the first level and UI-name of the layer/regionset as the name value. It supports multiple languages as any other maplayers in Oskari.

• dataprovider_id is a link to the data provider/organization that provides the layer

• additional configuration for the attributes column are:

  • regionIdTag: feature attribute that is the unique id for that region (like municipality/postal/zip code). This is used to map statistics data to a region
  • nameIdTag: feature attribute that has the name for the region (this is shown to the end-user as the region name)
  • featuresUrl: (deprecated) URL for corresponding WFS-service. It's used to read all the features to create a list of regions in the region set (this URL can be used to override the actual url-field. It was useful when we used both wms and wfs layers, but now it's considered deprecated and only the main url should be used.)

Note! Add the view permissions for the layer so users can see it.

Regionset as JSON resource

As of 1.46.0 Oskari version regionsets don't need to come from a WFS-service and having them as resource files under the webserver works as well:

1. Make your GeoJSON resource file available for the webapp container.

Store the file to the root resource directory for your web application (for example $JETTY_HOME/resources). When adding a resource file as a regionset layer configure featuresUrl in layer attributes as follows:

"resources://${path}"

Where ${path} is relative to the root resource directory in your web application. For example having the file in $JETTY_HOME/resources/regionsets/myfile.json would mean featuresUrl value of "resources://regionsets/myfile.json"

Note! The featuresUrl must start with "resources://" for the system to recognize that this layer is resource based and the file-extension MUST be '.json'.

2. The features describing the Regions in your GeoJSON resource need to have atleast two properties.

• One describing the id of the Region (the property name is configured as regionIdTag like in WFS-based config)
• Another describing the name of the Region (the property name is configured as nameIdTag like in WFS-based config)

Note! It's expected that the geometry in GeoJSON files is in the projection referenced in the srs_name column on oskari_maplayer for the regionset.

Adding a datasource

A datasource can be registered with a simple SQL:

INSERT INTO oskari_statistical_datasource(locale, config, plugin)
VALUES('{
    "en" : {
        "name":"Health and Welfare"
    }}',
    '{
        "url" : "http://www.sotkanet.fi/rest"
    }', 'SotkaNET');

Where:

• locale is a JSON with language code at the first level and UI-name of the datasource as the name value. It supports multiple languages like maplayers in Oskari.
• config is an adapter specific configuration that is used to give the adapter code hints how to process the datasource
• plugin is the ID for the adapter code to use for this datasource

Config can also include additional info about datasource and hints for sorting indicator data dimension values to be shown to user:

{
  "info" : {
    "url" : "https://moreinfo.here"
  },
  "hints" : {
    "dimensions" : [ {
      "id" : "year",
      "sort" : "DESC"
    }, {
      "id" : "gender",
      "default" : "total"
    }]
  }
}

Where id value will match the id of a data dimension item in indicator datamodel. Other keys affect the order of allowed values for that dimension. Sort (if present) will be done first with either DESC or ASC value. If default is present the matching allowed value will be moved as the first value in allowed values. You can use both, one or none.

The info-block is sent to the frontend code as is and the url (if provided) is shown as part of the attribution data for selected indicators.

Plugins/adapters

Plugins or adapters are used to interpret the statistics data API to a common format recognized by the Oskari frontend and map the statistics data to regions. There are a few API adapters available in Oskari and the oskari-server can be easily extended with additional adapters.

Eurostat

Eurostat is the statistical office of the European Union situated in Luxembourg. Its mission is to provide high quality statistics for Europe. They offer an API that uses SDMX and JSON-stat as dataformats.

Code: https://github.com/oskariorg/oskari-server/blob/develop/service-statistics-eurostat/src/main/java/fi/nls/oskari/statistics/eurostat/EurostatStatisticalDatasourceFactory.java

Datasource config:

{
    "url" : "http://ec.europa.eu/eurostat"
}

PxWeb

PxWeb is a widely used statistics software that offers an API for accessing the data.

Code: https://github.com/oskariorg/oskari-server/blob/develop/service-statistics-pxweb/src/main/java/fi/nls/oskari/control/statistics/plugins/pxweb/PxwebStatisticalDatasourceFactory.java

Datasource config:

{
    "url" : "http://some.pxweb.com/statdb",
    "regionKey" : "name of the attribute for the region id in stats data",
    "ignoredVariables": ["optional config", "any", "attributes", "that", "should", "be", "ignored"],
    "timeVariable": "Optional config for id of the variable that describes time like 'year'. This is used for time-series functionality.",
    "metadataFile": "/file/in/classpath.json (optional)"
}

The metadataFile configuration allows linking more metadata for indicators in the datasource like source for data, name/descriptions overrides for values from the API and configuring the type of data for selecting if it will be visualized as choropleth or point symbols by default etc. The value should point to a file in the server classpath. The format of the JSON is an array with objects like:

[{
    "code": "M408",
    "desc": {
        "fi": "Taajama-aste tarkoittaa taajamissa asuvien osuutta väestöstä, jonka sijainti tunnetaan. Taajamaksi määritellään kaikki vähintään 200 asukkaan rakennusryhmät, joissa rakennusten välinen etäisyys ei yleensä ole 200 metriä suurempi"
    },
    "source": {
        "fi": "Väestörakenne"
    },
    "isRatio": true,
    "base": 100,
    "min": -100,
    "max": 500,
    "decimalCount": 1,
    "timerange": {
        "start": "1987",
        "end": "2015"
    },
    "updated": "1.4.2016",
    "nextUpdate": "29.3.2017"
}, ...]

• code value is used to map the indicator between the metadata JSON and data from the PxWeb API and id should match the indicator id. This is used to map the metadata to the indicator. Everything else is optional.
• timerange can be used to configure indicator specific timeranges if they are all in the same .px file.
• updated and nextUpdate are shown in the UI with the indicator description.
• isRatio (optional boolean): true for choropleth, false for points
• base (optional number): with number value the classification distribution is set to Diverging. Missing or non-numeric value distribution defaults to Quantitative
• min and max describe the data but is not used by the frontend currently

SotkaNET

The Sotkanet Indicator Bank is an API provided by Finnish National Institute for Health and Welfare (THL). It is also used by National Land Survey of Finland to share statistics.

Code: https://github.com/oskariorg/oskari-server/blob/develop/service-statistics-sotka/src/main/java/fi/nls/oskari/control/statistics/plugins/sotka/SotkaStatisticalDatasourceFactory.java

Datasource config:

{
    "url" : "http://www.sotkanet.fi/rest",
    "timeVariable": "Optional config for id of the variable that describes time like 'year'. This is used for time-series functionality. Defaults to 'year' for SotkaNET if not configured."
}

Linking datasources and regionsets

Not all datasources have data for all of the regionsets so as the last step you need to link layers/regionsets that can be used with a given datasource.

INSERT INTO
    oskari_statistical_datasource_regionsets(datasource_id, layer_id, config)
VALUES(
    (SELECT id FROM oskari_statistical_datasource
        WHERE locale like '%Health and Welfare%'),
    (SELECT id FROM oskari_maplayer WHERE type='statslayer' AND name = 'mylayer'),
    '{}');

The config is an adapter specific configuration that can be used to pass information datasource specific information for the layer.

The current layer configuration options are listed below.

Eurostat

Doesn't use layer config, but could be used to detect which layer is used for which type of areas (NUTS 1,2,3).

PxWeb

Doesn't use layer config, but could be used to detect which layer is used for which type of areas in the statistics data or provide a fully qualified class name for doing the layer/statistics data mapping for more involved functionality.

SotkaNET

Uses config:

{
    "regionType" : "kunta"
}

The value of "regionType" should match the "category" value (like "kunta") in Sotkanet regions response (https://sotkanet.fi/rest/1.1/regions). The regionType value is case-insensitive. It's used to filter out indicators that the service has, but which don't have a regionset in the Oskari instance and as such can't be visualized in Oskari. Sotkanet data responses include data for all the regionsets and the same config is used to filter the data before it's passed to the frontend in Oskari.

10.1.3 Known issues

• performance is not as good as it should before data is cached.

• database configuration is cached on server startup so any changes to database for the statistics functionality will require a server restart. This includes for example regions in regionsets and datasources.

• There is no user interface to configure datasources so they need to be administrated with SQL

10.2 How to enable WFS-T (Content editor)

Tampere Oskari bundle content-editor

Extends Oskari functionality to support editing of wfs layer features and feature geometries.

Available in Oskari 1.36 or above.

10.2.1 Prerequisites

These instructions are based on jetty-8.1.16-oskari package.

10.2.2 Front-end

This bundle needs Oskari version 1.36 or above

10.2.3 Back-end

This functionality needs Oskari version 1.36 or above (oskari-map.war)

Database

Oskari database is needed without any special configuration

10.2.4 Installation

Create base folder i.e. Tampere

mkdir Tampere

Change directory to Tampere

cd Tampere

http://oskari.org/download

Download Oskari (http://oskari.org/build/server/jetty-8.1.16-oskari.zip) and extract archive to current (new created) directory

Goto jetty folder

cd jetty-8.1.16-oskari

10.2.5 Configuration

Add new permission type to oskari-ext.properties in [jetty-home]/resources folder

permission.types = EDIT_LAYER_CONTENT
permission.EDIT_LAYER_CONTENT.name.fi=Muokkaa tasoa
permission.EDIT_LAYER_CONTENT.name.en=Edit layer

Add bundle dynamically to correct roles in oskari-ext.properties. For example:

actionhandler.GetAppSetup.dynamic.bundles = admin-layerselector, admin-layerrights, admin-users, admin, content-editor
actionhandler.GetAppSetup.dynamic.bundle.content-editor.roles = Admin

Enter to jetty directory and run

java -jar start.jar

Use case in Oskari

Find out a WFS-layer where the service supports Transaction operation (WFS-T).
Add 'Edit' permission for that layer in Oskari Layer Rights functionality or through the layer administration UI (Edit permission type requires configuration on oskari-ext.properties to be shown).
Add the layer on the map in Oskari and the 'feature editor' functionality can now be accessed through the geoportals selected layers listing by selecting it from the tools for that layer.

10.2.6 Remarks

Editing is only available for geometry in EPSG:3067 CRS for time being.
Update SRID  to your wfst layer before editing, if it is 0 (default)

**( UPDATE [table] SET =ST_SetSRID([geometry field],3067) )**

WFS layer doesn't work e.g. for MapClick, if there is mixed SRID in the [table].
Mixed Srid could be checked with below sql:
**Select distinct(ST_SRID([geometry field])) as srid, count(*) from [table] group by srid;**

10.2.7 Known issues

Geometry type of geometry-column must be Geometry, MultiPoint, MultiLineString or MultiPolygon in Postgres wfs-t edit table (layer) for time being.

On the other words only MultiPoint, MultiLineString or MultiPolygon geometries are supported in Oskari feature edit (wfs-t).