This note gives a high-level overview of the EARTHSEQUENCING effort to provide a self-consistent database of ocean drilling data and drill-core meta data.

Rationale

As part of the EARTHSEQUENCING project, we are developing a set of databases that facilitate the processing, analysis and linking of various ocean drilling related data and information.
The main aim is to make it possible to quickly and consistently obtain data that are otherwise slow to access, and scattered across various existing databases.
The databases contain 

  • Information about DSDP/ODP/IODP Expeditions, Sites, Holes, Cores, Sections and related meta-data
  • Actual selected, and mainly multi-sensor track related, data aggregated by Hole and Analysis (partly accessible as data grid on https://paloz.marum.de/database or via a dedicated viewer DBViewer).
  • Selected processed digital image data, allowing fast generation of Site overviews, including core-correlation and splice visualisation (IODPCoreImage).
  • Information about some publicly available core-offset and splice tables (Core Affine Table, Splice Interval Table).

Eventually, further data, for example biostratigraphic occurrences, magnetostratigraphy, age models etc. can be linked into this structure.Here we give an overview about the Scientific Ocean Drilling core information part of the database, not the specific scientific measurements themselves (for that, see for example database overview).

Most data come from archived DSDP data, the old ODP Janus database, and the current IODP LIMS database, merged into a consistent format.

This project benefitted from the DFG funded PetaByte Storage Cluster (DFG Grant 405693605).

Overall Technical Infrastructure

The database system is not directly accessible externally due to security and "denial of service" considerations, but hooks are provided for human and machine readable access via web endpoints.
The databases are contained in a relational and filestore database system (Couchbase, Minio and several others). Public access is via an intermediary REST endpoint (Loopback).

Currently (February 2022), the database contains information about all DSDP/ODP/IODP expeditions conducted on the Glomar Challenger or Joides Resolution upto Expedition 385 (post-moratorium). Eventually MSP and Chikyu data might be added. 

In detail, the database entities consist of 

  • (Drilling) Platform, e.g. Glomar Challenger, Joides Resolution, Chikyu, MSP
  • Project (e.g. DSDP, ODP, IODP etc.)
  • Expedition
  • Site
  • Hole
  • Hole Occupation (e.g. Holes can be re-occupied)
  • Core
  • Core Section
  • Section Half
  • GeoLocation (to store co-ordinates of Holes)
  • TimeStamp
  • Person (e.g. CoChief Scientist, Expedition Project Manager)
  • CoreAffineTable
  • CoreAffineValue
  • SpliceIntervalTable
  • SpliceIntervalEntry

For the actual data storage, we use the two entities

  • AnalysisMapping (providing meta-information of data formats and import locations)
  • AnalysisHeader (per Hole, per Analysis compressed data and meta-data)

Data access

There are various ways to access these data, but as mentioned before, direct access is not available outside the private network for security reasons.

  1. For an overview of measurement data available, consult  https://paloz.marum.de/database. This is suitable for human download of text tables for individual holes and measurements.
  2. Various REST endpoints are available for machine-readable access. Please contact H. Pälike for further information, also to ensure overloading the current server. See below for a few examples of how this could be accessed:
  3. A current snapshot of the database (for OFFLINE viewing) is available as a macOS Application (see XXX).
  4. A number of pre-built applications allow access to these data also, for example DBViewer, IODPCoreImage.

Programmatic access is available via REST endpoints, for example 

A full list can be seen here:

REST endpoints

Offline Snapshot Viewer

A software for macOS only (sorry!) to view an offline snapshot of the database is available here:

CoreDataEditor-ODPDATA

Prior to running, this requires the installation of a database file into "~/Library/Application Support/" by unzipping the ~1GB file stored here: DATABASE.

Example:

Data Model Graph: