Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
448b9f4
update duckdb
Trophime May 5, 2026
ef15fc1
Migrate to node20
Trophime May 6, 2026
3619352
chore(web): upgrade node image from 18-bookworm to 20-trixie
Trophime May 6, 2026
5b8f8a0
update to support node20
Trophime May 6, 2026
5efeede
up for node 20
Trophime May 6, 2026
f466fa5
up to poetry 2.4.0 and trixie
Trophime May 6, 2026
1651c16
Remove uneeded file
Trophime May 6, 2026
c4b1255
update dev plans
Trophime May 6, 2026
867f55b
done with vue-cli migration
Trophime May 6, 2026
011824c
migrate to node 22
Trophime May 6, 2026
3f4e7e6
upgrade to ESLint 9
Trophime May 6, 2026
5939714
replace Vue.filter with global properties
Trophime May 6, 2026
af2721b
Merge pull request #29 from MagnetDB/eslint
Trophime May 7, 2026
e9d8a16
add pupitre/pigbrother files
Trophime May 7, 2026
79998db
add argumetns for setting datadir
Trophime May 7, 2026
237c596
Update poetry.lock: bump python-magnetgeo to 1.0.1
Trophime May 19, 2026
90c0968
up with to_duckdb demonstrator
Trophime Jun 11, 2026
ba2c5c1
update python_magnetsetup submodule
Trophime Jun 11, 2026
97ef6e1
add mysql connection demonstrator
Trophime Jun 11, 2026
56198f4
add missing js files
Trophime Jun 11, 2026
7e0e15d
add packages assesment yaml
Trophime Jun 15, 2026
f914b95
fix bug detected by M1 student -- populate operational data
Trophime Jun 18, 2026
f66c087
up with mysql connect example
Trophime Jun 18, 2026
996559a
fix polling mode -- except textual
Trophime Jun 18, 2026
06033c0
fix polling with textual -- suppress xlabel
Trophime Jun 18, 2026
a16f174
up mysql_connect doc
Trophime Jun 18, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .devcontainer/docker-compose-dev-traefik-ssl.yml
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,7 @@ services:
container_name: magnetdb-pgadmin
image: dpage/pgadmin4:8
environment:
PGADMIN_DEFAULT_EMAIL: christophe.trophime@lncmi.cnrs.fr
PGADMIN_DEFAULT_EMAIL: pgadmin@magnetdb-dev.local
PGADMIN_DEFAULT_PASSWORD: admin
# PGADMIN_CONFIG_SERVER_MODE: 'False'
# Environment variables for server auto-configuration
Expand Down
9 changes: 9 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -96,3 +96,12 @@ tmp/*
*.tar.gz
backups/

# duckdb
*.duckdb

*.old
*log

# direnv
.envrc*
*.my
4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM trophime/magnettools:bookworm-poetry-2.2.1
FROM trophime/magnettools:trixie-poetry-2.4.0

ARG USERNAME=feelpp
ARG USER_UID=1000
Expand Down Expand Up @@ -35,6 +35,6 @@ RUN echo "create local user $USERNAME (uid=${USER_UID}, gid=${USER_GID})" \
&& ssh-keyscan github.com >> /home/${USERNAME}/.ssh/known_hosts \
&& chown -R $USER_GID:$USER_UID /home/${USERNAME}/.ssh

ENV LD_LIBRARY_PATH="/usr/lib/x86_64-linux-gnu/MagnetTools/:$LD_LIBRARY_PATH"
ENV LD_LIBRARY_PATH="/usr/lib/x86_64-linux-gnu/MagnetTools/:${LD_LIBRARY_PATH:-}"
USER $USERNAME

4 changes: 2 additions & 2 deletions Dockerfile-dev
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM trophime/magnettools:bookworm-poetry-2.2.1
FROM trophime/magnettools:trixie-poetry-2.4.0

ARG USERNAME=feelpp
ARG USER_UID=1000
Expand Down Expand Up @@ -35,6 +35,6 @@ RUN echo "create local user $USERNAME (uid=${USER_UID}, gid=${USER_GID})" \
&& ssh-keyscan github.com >> /home/${USERNAME}/.ssh/known_hosts \
&& chown -R $USER_GID:$USER_UID /home/${USERNAME}/.ssh

ENV LD_LIBRARY_PATH="/usr/lib/x86_64-linux-gnu/MagnetTools/:$LD_LIBRARY_PATH"
ENV LD_LIBRARY_PATH="/usr/lib/x86_64-linux-gnu/MagnetTools/:${LD_LIBRARY_PATH:-}"
USER $USERNAME

2 changes: 1 addition & 1 deletion docker-compose-dev-traefik-ssl.yml
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ services:
- /bin/bash
- -c
- |
npm install --legacy-peer-deps
npm install
npm run serve
working_dir: /app
labels:
Expand Down
2 changes: 1 addition & 1 deletion docker-compose-traefik-ssl.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
version: '3'
services:
proxy:
image: traefik:3.3
image: traefik:v3.6.7
container_name: magnetdb-traefik
ports:
- "80:80"
Expand Down
11 changes: 0 additions & 11 deletions network_internal.conf

This file was deleted.

1,447 changes: 742 additions & 705 deletions poetry.lock

Large diffs are not rendered by default.

462 changes: 462 additions & 0 deletions prompts/JNL-G_DMP_full_Opidor.md

Large diffs are not rendered by default.

169 changes: 169 additions & 0 deletions prompts/MagnetDB_profile_README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
<!--
This file is intended to live at github.com/<MagnetDB-org>/.github → profile/README.md
It renders as the organization landing page.
-->

<!-- Optional: add a logo here -->
<!-- <p align="center"><img src="path/to/logo.svg" width="180" alt="MagnetDB logo"></p> -->

<p align="center">
<a href="https://anr.fr"><img src="https://anr.fr/typo3conf/ext/anr_skin/Resources/Public/assets/img/anr-logo-2021-complet.png" width="220" alt="ANR – Agence nationale de la recherche"></a>
&nbsp;&nbsp;
<img src="assets/jnlg-logo-1.png" width="80" alt="JNL-G project logo">
</p>

# MagnetDB

**A scientific data management and simulation platform for high-field magnet research.**

MagnetDB centralizes the configurations, operational data, simulation workflows and analytics of the high-field magnets operated at the [Laboratoire National des Champs Magnétiques Intenses](https://lncmi.cnrs.fr) (LNCMI), CNRS. LNCMI is the french magnet user facility located on 2 sites -- Grenoble (LNCMI-G) and Toulouse (LNCMI-T).

It is the foundation for the digital-twin work being carried out under the ANR JNL-G project (2026–2030) and integrates with the Feel++ scientific computing framework developed at [CEMOSIS](https://cemosis.fr).

The platform serves researchers, magnet designers, operators and lab directors with a single view of magnet configurations, real-time and historical operational data, simulation results, and predictive-maintenance signals.

---

## What's in this organization

The MagnetDB organization hosts the open-source components of the platform. Restricted components (in particular parts of MagnetTools) are kept in private repositories; built artifacts for those are distributed via the LNCMI internal apt depot and the `feelpp/magnettools` DockerHub image.

### Core platform

| Repository | Purpose |
|---|---|
| **`python_magnetdb`** | Django + FastAPI backend, PostgreSQL schema, REST API, background workers, S3/MinIO integration |
| **`magnetdb-frontend`** | Web frontend — currently migrating from Vue 2 to React 18 + TypeScript + Vite |
| **`python_magnetapi`** | Lightweight client library for the MagnetDB API |

### Geometry, mesh and CAD

| Repository | Purpose |
|---|---|
| **`python_magnetgeo`** | Polymorphic geometry definitions for LNCMI-G resistive and superconductor magnets |
| **`python_magnetgmsh`** | Mesh generation from `python_magnetgeo` definitions via Gmsh |
| **`hifimagnet-salome`** | CAD and Mesh Generation with [SALOME](https://www.salome-platform.org) tools (requires a valid **MeshGems** license) |

### Simulation setup and execution

| Repository | Purpose |
|---|---|
| **`python_magnetsetup`** | Templating engine that turns a MagnetDB configuration into a Feel++ / HiFiMagnet simulation case |
| **`python_magnetworkflow`** | Orchestrated workflow: setup → CAD → mesh → run → archive |
| **`hifimagnet-paraview`** | ParaView plugins and post-processing scripts |

### Operational data and analysis

| Repository | Purpose |
|---|---|
| **`python_magnetrun`** | Parsing, curation and analysis of magnets operational data from the different control and monitoring systems -- namely Pupitre, PigBrother and Hybrid MCS recordings following internal LNCMI nomenclature |
| **`magnet-scipy`** | SciPy-based analytics for magnet operational data |
| **`python_magnetcooling`** | Cooling-loop and thermal-hydraulic modelling helpers |

### C++ tools (with Python bindings)

| Repository | Purpose |
|---|---|
| **`python_magnettools`** | Python bindings to the C++ MagnetTools library (analytical magnet calculations) |

---

## How the pieces fit together

```
┌───────────────────────────────────────────────────┐
│ MagnetDB (Django + FastAPI) │
│ PostgreSQL · MinIO (S3) · Celery · LemonLDAP SSO │
└───────────────────────────────────────────────────┘
▲ ▲
┌──────────────────┘ └──────────────────┐
│ │
operational data simulation pipeline
│ │
python_magnetrun python_magnetsetup
│ │ │
curation Feel++ │ HiFiMagnet
ETL │ │
stats Orchestrated workflows
anomaly │ │
detection │ python_magnetgmsh
│ │ │
▼ ▼ ▼
Pupitre · PigBrother · Hybrid CAD · mesh · results · provenance
(MCS recordings, sshfs) (attachments in MinIO)
```

The platform follows a three-layer operational data architecture:

- **Raw** (sshfs, LNCMI-internal) → **Curated Parquet on S3** (canonical interchange layer) → **Derived statistics in PostgreSQL + S3**.

---

## Getting started

- **Documentation**: see the docs site of the relevant repository (each component ships its own Sphinx / Antora site)
- **Demo stack**: a minimal Docker Compose setup (postgres + redis + minio + api + jupyter) is available in `python_magnetdb` for students and external evaluators
- **Container images**: published to GitHub Container Registry (`ghcr.io/<org>/...`)
- **HPC**: Apptainer / Singularity images are available for SLURM / PBS / SGE clusters

If you are joining the project as a student or contributor, start with `python_magnetdb` and the demo Docker Compose stack — it gives you a working MagnetDB instance with a curated seed dataset in under five minutes.

> [!NOTE] The demo stack is intended for evaluation and development purposes. It is not production-ready and should not be used to deploy a real MagnetDB instance.

> [!NOTE] Container images may not be actually published to GHCR until the first stable release (v1.0.0) is tagged.

---

## Related projects

- **[Feel++](https://github.com/feelpp/feelpp)** — finite-element framework used by HiFiMagnet for non-linear multiphysics magnet simulation
- **Scimba** — scientific machine-learning framework used in the JNL-G project for ROM and Neural Galerkin methods
- **JNL-G (ANR AAPG2025)** — Digital twin of the LNCMI-G installation; the project under which MagnetDB is being extended into a full digital-twin platform
- **FASUM (ANR Equipex+)** — 40 T all-superconducting user magnet at LNCMI-G; MagnetDB ingests its operational data
- **EMFL** — European Magnetic Field Laboratory; LNCMI-G is one of three EMFL member facilities

---

## Contributing

Contributions are welcome. Each repository has its own `CONTRIBUTING.md` describing coding standards, testing, and the pull-request workflow.

- Issues and pull requests are tracked per repository
- Larger architectural discussions happen in `python_magnetdb` Discussions
- Releases follow [semantic versioning](https://semver.org/)
- Tagged releases are archived on [Software Heritage](https://www.softwareheritage.org/) and [Zenodo](https://zenodo.org/) with a DOI

---

## Data management

The data managed by MagnetDB in the context of the JNL-G project is described in the project's Data Management Plan. **JNL-G's data are operational data of the LNCMI-G installation — recordings produced by the magnet, power and cooling control systems** while a magnet is energized. User experimental data (measurements performed by external user teams on their own samples) are out of scope and are governed by the [EMFL DMP](https://emfl.eu/).

---

## Funding and acknowledgments

MagnetDB is developed at LNCMI (CNRS UPR 3228) in collaboration with CEMOSIS (Université de Strasbourg). Its development has received support from:

- **ANR AAPG2025 JNL-G** — *Digital twin of the LNCMI-G*
- **H2020 ISABEL** (Grant Agreement No 871106) — *Improving the sustainability of the European Magnetic Field Laboratory*
- **ANR Equipex+ FASUM** (ANR-21-ESRE-0027) — *Forty Tesla Superconducting User Magnet*
- **CNRS** institutional support to LNCMI as a national research infrastructure

<p>
<a href="https://anr.fr"><img src="https://anr.fr/typo3conf/ext/anr_skin/Resources/Public/assets/img/anr-logo-2021-complet.png" height="50" alt="ANR – Agence nationale de la recherche"></a>
&nbsp;&nbsp;
<img src="assets/jnlg-logo-1.png" height="50" alt="JNL-G project logo">
</p>

---

## Contact

- **Lead developer**: Christophe Trophime — *christophe.trophime@lncmi.cnrs.fr*
- **LNCMI Grenoble**: 25 avenue des Martyrs, 38000 Grenoble, France — [lncmi.cnrs.fr](https://lncmi.cnrs.fr)
- **Issues**: please open them in the relevant repository of this organization

---

<sub>MagnetDB is open-source software. Public components are released under the Apache-2.0 or LGPL-3.0 license (see each repository for its exact terms). Documentation and demonstration datasets are released under CC0 or CC-BY-4.0.</sub>
Loading