Metadata Builder

The Metadata Builder converts a dataset’s structure, filenames, existing sidecars, and readable image headers into a transparent set of metadata inventories, BIDS-oriented templates, validation summaries, and review tables.

It can be used after creating a new dataset in NIM Studio or independently to inspect and curate an existing raw, derivative, or study-level dataset.

The Metadata Builder does not assume that automatically inferred values are complete or scientifically correct. All generated modality templates are marked for manual review, and missing required or recommended fields remain visible before export.

Overview

The Metadata Builder scans a selected dataset recursively and summarizes its participants, sessions, files, modalities, tasks, acquisitions, runs, suffixes, existing JSON files, NIfTI files, and readable DICOM metadata.

During the scan, NIM Studio combines information from:

  • Folder structure

  • BIDS-style filenames

  • Existing sidecar JSON files

  • NIfTI headers, when nibabel is installed

  • DICOM headers, when pydicom is installed

The resulting metadata are associated with their detected source so that users can distinguish values inferred from filenames, folder organization, existing JSON files, NIfTI headers, and DICOM headers. :contentReference[oaicite:0]{index=0}

Supported Dataset Contexts

The Metadata Builder can scan:

  • Raw BIDS datasets

  • Derivative datasets

  • Study-level dataset roots

  • Partially organized BIDS-like datasets

  • Existing datasets created outside NIM Studio

The selected root is classified as raw, derivative, or study using its directory context and detected contents. For derivative datasets, the application also pre-fills the pipeline name using the selected folder name when no value has been entered. :contentReference[oaicite:1]{index=1}

Dataset Description

The module provides an editable interface for constructing dataset_description.json.

Supported fields include:

Required fields

  • Name

  • BIDSVersion

Dataset type

  • raw

  • derivative

  • study

Derivative provenance fields

For derivative datasets, users can document:

  • GeneratedBy.Name

  • Pipeline version

  • Pipeline or processing description

  • Source dataset URL

  • Source dataset DOI

  • Source dataset version

A derivative dataset cannot save a selected dataset_description.json unless GeneratedBy.Name has been provided. :contentReference[oaicite:2]{index=2} :contentReference[oaicite:3]{index=3}

When a dataset has been scanned, NIM Studio may also add a structural summary containing the number of detected participants, participant-session rows, files, and datatypes. It records that the metadata templates were generated by NIM Studio and still require user verification. :contentReference[oaicite:4]{index=4}

Participant Management

The Metadata Builder creates participant-level summaries from detected sub-<label> folders and filenames.

The generated participants.tsv can include:

  • Participant identifier

  • Detected sessions

  • Detected datatypes or modalities

  • Detected tasks

  • Detected acquisition labels

  • Detected runs

  • Number of files associated with each participant

Researchers may also define custom participant columns, such as:

  • age

  • sex

  • diagnosis

  • group

  • site

Custom columns are created as empty fields for later completion. NIM Studio does not infer clinical or demographic values from the dataset. :contentReference[oaicite:5]{index=5}

The accompanying participants.json documents the meaning of each generated column, including researcher-defined variables.

Session Management

When sessions are detected, the module can generate sessions.tsv and sessions.json.

Each participant-session row may contain:

  • Participant identifier

  • Session identifier

  • Datatypes detected in that session

  • Tasks detected in that session

  • Number of files detected

The current implementation generates a combined session table at the selected output root. It does not automatically create separate BIDS sub-<label>_sessions.tsv files within every participant directory.

Scan Inventory

The generated scans.tsv acts as a dataset-wide file inventory.

For each scanned file it may include:

  • Relative filename

  • Participant identifier

  • Session identifier

  • Datatype

  • Suffix

  • Task

  • Acquisition

  • Run

  • Description

  • Space

  • Extension

  • Whether an existing JSON sidecar was detected

  • Missing required metadata fields

  • Manual-review status

The corresponding scans.json describes these columns.

This inventory is broader than the standard participant-level BIDS scans.tsv convention and should therefore be treated as a NIM Studio dataset audit and metadata-curation table unless it is manually adapted to a specific BIDS scope. :contentReference[oaicite:6]{index=6}

Header Extraction

NIfTI Metadata

When nibabel is available, NIM Studio attempts to read:

  • Image dimensions

  • Voxel size

  • Temporal zoom as a proposed RepetitionTime

  • NIfTI data type

Header extraction is best-effort. Files that cannot be read are skipped without preventing the remainder of the scan. :contentReference[oaicite:7]{index=7}

DICOM Metadata

When pydicom is available, NIM Studio reads DICOM headers without loading pixel data and maps available fields into BIDS-oriented names.

Examples include:

  • Manufacturer

  • Scanner model

  • Device serial number

  • Software version

  • Magnetic-field strength

  • Coil name

  • Institution information

  • Sequence and protocol information

  • Repetition time

  • Echo time

  • Inversion time

  • Flip angle

  • Pixel bandwidth

  • Slice thickness

  • Image rows and columns

Limited PET-oriented fields may also be extracted when they are available, including tracer name, injected radioactivity, radionuclide half-life, and start time. Header availability varies across vendors and export formats. :contentReference[oaicite:8]{index=8}

Existing Sidecars

When a matching JSON sidecar exists, NIM Studio reads it and incorporates its values into the metadata record.

The current merge order is:

  1. Filename-derived metadata

  2. NIfTI-header metadata

  3. DICOM-header metadata

  4. Existing JSON metadata

Later sources replace earlier values when the same field is present. Consequently, existing sidecar values take precedence over automatically inferred header or filename values. :contentReference[oaicite:9]{index=9} :contentReference[oaicite:10]{index=10}

Modality Sidecar Templates

The Metadata Builder groups detected files by likely sidecar scope and creates JSON templates such as:

T1w.json
task-rest_bold.json
task-nback_events.json
dwi.json
eeg.json

Templates are populated from representative metadata values detected across the dataset.

Each generated template contains a NIMStudioMetadataStatus section recording:

  • That manual review is required

  • Detected datatype and suffix

  • Number of files represented

  • Required fields

  • Recommended fields

  • Missing required fields

  • Missing recommended fields

  • Sources of populated values

These templates are review aids. They are not automatically certified as complete BIDS sidecars. :contentReference[oaicite:11]{index=11}

Validation Summary

NIM Studio compares detected metadata against an internal registry of selected required and recommended fields.

The current registry includes configurations for several common modalities, including:

  • Anatomical MRI

  • Functional MRI and SBRef

  • Diffusion MRI

  • Field maps

  • ASL

  • PET

  • EEG

  • MEG

  • iEEG

  • Physiological recordings

  • MR spectroscopy

The validation view reports how often required and recommended fields are missing across scanned files. Missing recommended fields do not prevent export. Missing required modality fields trigger a warning, but users may still export the files explicitly as review templates. :contentReference[oaicite:12]{index=12} :contentReference[oaicite:13]{index=13}

This is a targeted metadata completeness assessment based on the NIM Studio registry. It is not a complete implementation of the official BIDS validation rules.

Metadata Master Table

The module can generate a merged metadata table in CSV or XLSX format.

Each row represents a scanned file and may contain:

  • Dataset root and dataset type

  • Participant and session identifiers

  • BIDS entities

  • Filename and relative path

  • Existing-sidecar status

  • Scanner and acquisition information

  • Timing parameters

  • Image dimensions and voxel size

  • PET metadata

  • Metadata source information

  • Required and recommended fields

  • Missing fields

  • Manual-review status

The table provides a centralized metadata inventory suitable for review, harmonization, quality control, and preparation for later curation. :contentReference[oaicite:14]{index=14}

Available Outputs

Users can preview and selectively save:

  • dataset_description.json

  • participants.tsv

  • participants.json

  • sessions.tsv

  • sessions.json

  • scans.tsv

  • scans.json

  • Modality sidecar JSON templates

  • metadata_master.csv

  • metadata_master.xlsx

XLSX export requires openpyxl. NIfTI and DICOM header extraction require nibabel and pydicom respectively. If these optional packages are not installed, the remaining metadata functions can still be used.

Before saving, users choose the destination directory. Existing files with the same names are written to that location, so users should inspect the selected folder and retain backups before exporting into an existing dataset. :contentReference[oaicite:15]{index=15}

What the Metadata Builder Does Not Do

The current beta does not:

  • Guarantee that a dataset is fully BIDS-compliant.

  • Replace the official BIDS Validator.

  • Convert DICOM images into NIfTI.

  • Determine scientifically correct acquisition parameters when headers are absent or ambiguous.

  • Resolve every BIDS inheritance relationship automatically.

  • Determine the correct scope for every generated sidecar.

  • Generate complete events.tsv content from task logs.

  • Infer demographic, clinical, or protected participant information.

  • Validate consent, ethics approval, anonymization, or GDPR compliance.

  • Remove identifying DICOM fields.

  • Guarantee that a NIfTI temporal zoom represents the intended BIDS RepetitionTime in every case.

  • Harmonize conflicting metadata across scanners, sites, sessions, or cohorts.

  • Preserve existing files automatically when exporting to the same location.

  • Make missing required fields scientifically valid by inserting placeholders.

Beta Safety Notice

Generated metadata should initially be treated as draft documentation and curation material.

During the beta phase, users should:

  • Test the module on non-critical or backed-up data.

  • Review every inferred field.

  • Compare header-derived values with acquisition protocols.

  • Keep identifying and confidential metadata within approved environments.

  • Export into a separate review directory when working with established datasets.

  • Validate the final dataset independently before analysis, publication, or sharing.