RDF master and reference data about crops

This project addresses the challenge of fragmented agricultural crop data within the Swiss federal administration, where essential systems¹ all use separate, non-harmonized crop terminologies. This lack of a "single source of truth" creates significant integration hurdles for digital tools.

In this project, we propose a unified master data system for crops and crop-related objects. The repository implements a sustainable solution by using a dedicated RDF ontology (crops ontology) and a graph database on LINDAS. This approach first connects (or "maps") the various crop terms from the different systems, creating a unified, machine-readable master data system that can be queried centrally. This graph not only allows for complex queries across formerly siloed data but also provides the stable, versioned foundation for the long-term, step-by-step harmonization of crop data across the Swiss agricultural sector. Click here to search for crops in the graph.

Warning

This project is still work in progress.

Inspect the hierarchy

This hierarchy viewer allows you to visually inspect the hierarchical relationships.

Inspect the ontology

Inspect the ontology using WebVOWL here or read its turtle file here.

Run the data processing and LINDAS integration pipeline

The data integration pipeline uses all the R and python scripts in the /scripts folder. The entire pipeline can be triggered with:

1. Add variables to .env

   USER=lindas-foag
   PASSWORD=********
   GRAPH=https://lindas.admin.ch/foag/crops
   ENDPOINT=https://stardog.cluster.ldbar.ch/lindas
   EPPO=********

2. Start a virtual environment and install libraries:

   python -m venv venv
   source venv/bin/activate  # On Windows use: venv\Scripts\activate
   pip install -r scripts/Python/requirements.txt

3. Run the ETL pipeline sh scripts/graph-processing.sh

4. Choose whether or not to generate and upload geodata.ttl, which enables queries and depiction of crop areas.

5. Check out the results on LINDAS.

Data mapping and unification

Data about crops is often sourced from various systems, which can lead to duplicate entries for the same real-world concept. To create a clean and unified dataset, we employ a mapping process to consolidate these duplicates.

This consolidation is defined in the rdf/mapping.ttl file. It uses the standard OWL property owl:sameAs to declare that two URIs refer to the same entity.

For example, consider the following statement:

@prefix : <https://agriculture.ld.admin.ch/crops/> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .

:950 owl:sameAs :555 .

This statement establishes :950 as the canonical (master) URI and :555 as the duplicate. During the data integration pipeline, the scripts/reason.py script processes this mapping. In this example, all triples that use :555 as a subject or object are automatically rewritten to use :950 instead. Crucially, to avoid conflicting information, the canonical entity :950 first loses all its properties for names and descriptions (specifically schema:name and schema:description). This ensures that the descriptive properties from the merged entity (:555) are cleanly transferred, creating a single, consistent record for the crop under the URI :950.

Explore the graph on LINDAS

You can query the crop master data system using SPARQL.

Here's an example SPARQL query that gets you all cultivation type URIs and labels in German:

PREFIX schema: <http://schema.org/>
PREFIX : <https://agriculture.ld.admin.ch/crops/>
SELECT *
WHERE {
  ?crop a :CultivationType .
  ?crop schema:name ?name .
  FILTER(LANG(?name)="de")
}
ORDER BY ?name

Footnotes

1. For example AGIS (direct payments), GRUD (fertilization), PSM registry (plant protection), ProVar (varieties), PGREL-NIS (gene bank) and others. ↩