Skip to content

Commit ae134fe

Browse files
chore(pre-commit): use mdformat to format Markdown (#941)
Follow up deepmodeling/deepmd-kit@69eb0c3, use mdformat to format Markdown. - Replace blacken-docs with mdformat <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated pre-commit config to switch to an mdformat-based markdown formatter, add exclusions for tests, and include several mdformat plugins. * **Documentation** * Added detailed BondOrderSystem docs with usage examples and sanitization/charge guidance. * Expanded and clarified multiple system docs (mixed, multi, system) with examples and formatting improvements. * Minor formatting/readability edits to AGENTS.md and dpdata-cli README/SKILL.md. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: njzjz-bot <njzjz-bot@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1 parent 22bba14 commit ae134fe

8 files changed

Lines changed: 170 additions & 102 deletions

File tree

.pre-commit-config.yaml

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -34,8 +34,18 @@ repos:
3434
hooks:
3535
- id: velin
3636
args: ["--write"]
37-
# Python inside docs
38-
- repo: https://github.com/asottile/blacken-docs
39-
rev: 1.20.0
37+
# markdown
38+
- repo: https://github.com/hukkin/mdformat
39+
rev: 1.0.0
4040
hooks:
41-
- id: blacken-docs
41+
- id: mdformat
42+
exclude: "^tests/.*$"
43+
additional_dependencies:
44+
# - mdformat-myst==0.3.0
45+
# See https://github.com/executablebooks/mdformat-myst/issues/13
46+
- "git+https://github.com/njzjz-bothub/mdformat-myst@d9c414e#egg=mdformat-myst"
47+
- mdformat-ruff==0.1.3
48+
- mdformat-web==0.2.0
49+
- mdformat-config==0.2.1
50+
- mdformat-beautysh==1.0.0
51+
- mdformat-gfm-alerts==2.0.0

AGENTS.md

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,51 +7,60 @@ Always reference these instructions first and fallback to search or bash command
77
## Working Effectively
88

99
- **Bootstrap and install the repository:**
10+
1011
- `cd /home/runner/work/dpdata/dpdata` (or wherever the repo is cloned)
1112
- `uv pip install -e .` -- installs dpdata in development mode with core dependencies (numpy, scipy, h5py, monty, wcmatch)
1213
- Test installation: `dpdata --version` -- should show version like "dpdata v0.1.dev2+..."
1314

1415
- **Run tests:**
16+
1517
- `cd tests && python -m unittest discover` -- runs all 1826 tests in ~10 seconds. NEVER CANCEL.
1618
- `cd tests && python -m unittest test_<module>.py` -- run specific test modules (individual modules take ~0.5 seconds)
1719
- `cd tests && coverage run --source=../dpdata -m unittest discover && coverage report` -- run tests with coverage
1820

1921
- **Linting and formatting:**
22+
2023
- Install ruff: `uv pip install ruff`
2124
- `ruff check dpdata/` -- lint the main package (takes ~1 second)
2225
- `ruff format dpdata/` -- format code according to project style
2326
- `ruff check --fix dpdata/` -- auto-fix linting issues where possible
2427

2528
- **Pre-commit hooks:**
29+
2630
- Install: `uv pip install pre-commit`
2731
- `pre-commit run --all-files` -- run all hooks on all files
2832
- Hooks include: ruff linting/formatting, trailing whitespace, end-of-file-fixer, yaml/json/toml checks
2933

3034
## Validation
3135

3236
- **Always test CLI functionality after making changes:**
37+
3338
- `dpdata --help` -- ensure CLI still works
3439
- `dpdata --version` -- verify version is correct
3540
- Test a basic conversion if sample data is available
3641

3742
- **Always run linting before committing:**
43+
3844
- `ruff check dpdata/` -- ensure no new linting errors
3945
- `ruff format dpdata/` -- ensure code is properly formatted
4046

4147
- **Run relevant tests for your changes:**
48+
4249
- For format-specific changes: `cd tests && python -m unittest test_<format>*.py`
4350
- For core system changes: `cd tests && python -m unittest test_system*.py test_multisystems.py`
4451
- For CLI changes: `cd tests && python -m unittest test_cli.py` (if exists)
4552

4653
## Build and Documentation
4754

4855
- **Documentation:**
56+
4957
- `cd docs && make help` -- see all available build targets
5058
- `cd docs && make html` -- build HTML documentation (requires additional dependencies)
5159
- Documentation source is in `docs/` directory using Sphinx
5260
- **NOTE:** Full docs build requires additional dependencies like `deepmodeling-sphinx` that may not be readily available
5361

5462
- **Package building:**
63+
5564
- Uses setuptools with pyproject.toml configuration
5665
- `uv pip install build && python -m build` -- create source and wheel distributions
5766
- Version is managed by setuptools_scm from git tags
@@ -61,6 +70,7 @@ Always reference these instructions first and fallback to search or bash command
6170
The following are outputs from frequently run commands. Reference them instead of re-running to save time.
6271

6372
### Repository structure
73+
6474
```
6575
/home/runner/work/dpdata/dpdata/
6676
├── dpdata/ # Main package code
@@ -82,39 +92,46 @@ The following are outputs from frequently run commands. Reference them instead o
8292
```
8393

8494
### Key dependencies
95+
8596
- Core: numpy>=1.14.3, scipy, h5py, monty, wcmatch
8697
- Optional: ase (ASE integration), parmed (AMBER), pymatgen (Materials Project), rdkit (molecular analysis)
8798
- Testing: unittest (built-in), coverage
8899
- Linting: ruff
89100
- Docs: sphinx with various extensions
90101

91102
### Test timing expectations
103+
92104
- Full test suite: ~10 seconds (1826 tests). NEVER CANCEL.
93105
- Individual test modules: ~0.5 seconds
94106
- Linting with ruff: ~1 second
95107
- Documentation build: ~30 seconds
96108

97109
### Common workflows
110+
98111
1. **Adding a new format:**
112+
99113
- Create module in `dpdata/<format>/`
100114
- Implement format classes inheriting from appropriate base classes
101115
- Add tests in `tests/test_<format>*.py`
102116
- Register format in the plugin system
103117

104-
2. **Fixing bugs:**
118+
1. **Fixing bugs:**
119+
105120
- Write test that reproduces the bug first
106121
- Make minimal fix to pass the test
107122
- Run full test suite to ensure no regressions
108123
- Run linting to ensure code style compliance
109124

110-
3. **CLI changes:**
125+
1. **CLI changes:**
126+
111127
- Modify `dpdata/cli.py`
112128
- Test with `dpdata --help` and specific commands
113129
- Add/update tests if needed
114130

115131
## Troubleshooting
116132

117133
- **Installation timeouts:** Network timeouts during `uv pip install` are common. If this occurs, try:
134+
118135
- Individual package installation: `uv pip install numpy scipy h5py monty wcmatch`
119136
- Use `--timeout` option: `uv pip install --timeout 300 -e .`
120137
- Verify existing installation works: `dpdata --version` should work even if reinstall fails

docs/systems/bond_order_system.md

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
1-
21
## BondOrderSystem
2+
33
A new class {class}`BondOrderSystem <dpdata.BondOrderSystem>` which inherits from class {class}`System <dpdata.System>` is introduced in dpdata. This new class contains information of chemical bonds and formal charges (stored in `BondOrderSystem.data['bonds']`, `BondOrderSystem.data['formal_charges']`). Now BondOrderSystem can only read from .mol/.sdf formats, because of its dependency on rdkit (which means rdkit must be installed if you want to use this function). Other formats, such as pdb, must be converted to .mol/.sdf format (maybe with software like open babel).
4+
45
```python
56
import dpdata
67

@@ -11,8 +12,10 @@ system_2 = dpdata.BondOrderSystem(
1112
"tests/bond_order/methane.sdf", fmt="sdf"
1213
) # read from .sdf file
1314
```
15+
1416
In sdf file, all molecules must be of the same topology (i.e. conformers of the same molecular configuration).
1517
`BondOrderSystem <dpdata.BondOrderSystem>` also supports initialize from a {class}`rdkit.Chem.rdchem.Mol` object directly.
18+
1619
```python
1720
from rdkit import Chem
1821
from rdkit.Chem import AllChem
@@ -25,21 +28,26 @@ system = dpdata.BondOrderSystem(rdkit_mol=mol)
2528
```
2629

2730
### Bond Order Assignment
31+
2832
The {class}`BondOrderSystem <dpdata.BondOrderSystem>` implements a more robust sanitize procedure for rdkit Mol, as defined in {class}`dpdata.rdkit.santizie.Sanitizer`. This class defines 3 level of sanitization process by: low, medium and high. (default is medium).
29-
+ low: use `rdkit.Chem.SanitizeMol()` function to sanitize molecule.
30-
+ medium: before using rdkit, the programm will first assign formal charge of each atom to avoid inappropriate valence exceptions. However, this mode requires the rightness of the bond order information in the given molecule.
31-
+ high: the program will try to fix inappropriate bond orders in aromatic hetreocycles, phosphate, sulfate, carboxyl, nitro, nitrine, guanidine groups. If this procedure fails to sanitize the given molecule, the program will then try to call `obabel` to pre-process the mol and repeat the sanitization procedure. **That is to say, if you wan't to use this level of sanitization, please ensure `obabel` is installed in the environment.**
32-
According to our test, our sanitization procedure can successfully read 4852 small molecules in the PDBBind-refined-set. It is necessary to point out that the in the molecule file (mol/sdf), the number of explicit hydrogens has to be correct. Thus, we recommend to use
33-
`obabel xxx -O xxx -h` to pre-process the file. The reason why we do not implement this hydrogen-adding procedure in dpdata is that we can not ensure its correctness.
33+
34+
- low: use `rdkit.Chem.SanitizeMol()` function to sanitize molecule.
35+
- medium: before using rdkit, the programm will first assign formal charge of each atom to avoid inappropriate valence exceptions. However, this mode requires the rightness of the bond order information in the given molecule.
36+
- high: the program will try to fix inappropriate bond orders in aromatic hetreocycles, phosphate, sulfate, carboxyl, nitro, nitrine, guanidine groups. If this procedure fails to sanitize the given molecule, the program will then try to call `obabel` to pre-process the mol and repeat the sanitization procedure. **That is to say, if you wan't to use this level of sanitization, please ensure `obabel` is installed in the environment.**
37+
According to our test, our sanitization procedure can successfully read 4852 small molecules in the PDBBind-refined-set. It is necessary to point out that the in the molecule file (mol/sdf), the number of explicit hydrogens has to be correct. Thus, we recommend to use
38+
`obabel xxx -O xxx -h` to pre-process the file. The reason why we do not implement this hydrogen-adding procedure in dpdata is that we can not ensure its correctness.
3439

3540
```python
3641
import dpdata
3742

3843
for sdf_file in glob.glob("bond_order/refined-set-ligands/obabel/*sdf"):
3944
syst = dpdata.BondOrderSystem(sdf_file, sanitize_level="high", verbose=False)
4045
```
46+
4147
### Formal Charge Assignment
48+
4249
BondOrderSystem implement a method to assign formal charge for each atom based on the 8-electron rule (see below). Note that it only supports common elements in bio-system: B,C,N,O,P,S,As
50+
4351
```python
4452
import dpdata
4553

docs/systems/mixed.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,13 +10,15 @@ This also helps to mixture the type information together for model training with
1010
Here are examples using `deepmd/npy/mixed` format:
1111

1212
- Dump a MultiSystems into a mixed type numpy directory:
13+
1314
```python
1415
import dpdata
1516

1617
dpdata.MultiSystems(*systems).to_deepmd_npy_mixed("mixed_dir")
1718
```
1819

1920
- Load a mixed type data into a MultiSystems:
21+
2022
```python
2123
import dpdata
2224

docs/systems/multi.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,16 @@
11
# `MultiSystems`
22

3-
The Class {class}`dpdata.MultiSystems` can read data from a dir which may contains many files of different systems, or from single xyz file which contains different systems.
3+
The Class {class}`dpdata.MultiSystems` can read data from a dir which may contains many files of different systems, or from single xyz file which contains different systems.
44

5-
Use {meth}`dpdata.MultiSystems.from_dir` to read from a directory, {class}`dpdata.MultiSystems` will walk in the directory
6-
Recursively and find all file with specific file_name. Supports all the file formats that {class}`dpdata.LabeledSystem` supports.
5+
Use {meth}`dpdata.MultiSystems.from_dir` to read from a directory, {class}`dpdata.MultiSystems` will walk in the directory
6+
Recursively and find all file with specific file_name. Supports all the file formats that {class}`dpdata.LabeledSystem` supports.
77

88
Use {meth}`dpdata.MultiSystems.from_file` to read from single file. Single-file support is available for the `quip/gap/xyz` and `ase/structure` formats.
99

1010
For example, for `quip/gap xyz` files, single .xyz file may contain many different configurations with different atom numbers and atom type.
1111

1212
The following commands relating to {class}`dpdata.MultiSystems` may be useful.
13+
1314
```python
1415
# load data
1516

@@ -40,6 +41,7 @@ xyz_multi_systems.to_deepmd_raw("./my_deepmd_data/")
4041
```
4142

4243
You may also use the following code to parse muti-system:
44+
4345
```python
4446
from dpdata import LabeledSystem, MultiSystems
4547
from glob import glob

docs/systems/system.md

Lines changed: 41 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,84 +1,103 @@
11
# `System` and `LabeledSystem`
22

33
This section gives some examples on how dpdata works. Firstly one needs to import the module in a python 3.x compatible code.
4+
45
```python
56
import dpdata
67
```
8+
79
The typicall workflow of `dpdata` is
810

911
1. Load data from vasp or lammps or deepmd-kit data files.
10-
2. Manipulate data
11-
3. Dump data to in a desired format
12-
12+
1. Manipulate data
13+
1. Dump data to in a desired format
1314

1415
### Load data
16+
1517
```python
1618
d_poscar = dpdata.System("POSCAR", fmt="vasp/poscar")
1719
```
20+
1821
or let dpdata infer the format (`vasp/poscar`) of the file from the file name extension
22+
1923
```python
2024
d_poscar = dpdata.System("my.POSCAR")
2125
```
26+
2227
The number of atoms, atom types, coordinates are loaded from the `POSCAR` and stored to a data {class}`System <dpdata.System>` called `d_poscar`.
2328
A data {class}`System <dpdata.System>` (a concept used by [deepmd-kit](https://github.com/deepmodeling/deepmd-kit)) contains frames that has the same number of atoms of the same type. The order of the atoms should be consistent among the frames in one {class}`System <dpdata.System>`.
2429
It is noted that `POSCAR` only contains one frame.
2530
If the multiple frames stored in, for example, a `OUTCAR` is wanted,
31+
2632
```python
2733
d_outcar = dpdata.LabeledSystem("OUTCAR")
2834
```
35+
2936
The labels provided in the `OUTCAR`, i.e. energies, forces and virials (if any), are loaded by {class}`LabeledSystem <dpdata.LabeledSystem>`. It is noted that the forces of atoms are always assumed to exist. {class}`LabeledSystem <dpdata.LabeledSystem>` is a derived class of {class}`System <dpdata.System>`.
3037

3138
The {class}`System <dpdata.System>` or {class}`LabeledSystem <dpdata.LabeledSystem>` can be constructed from the [supported file formats](../formats.rst) with the `format key` in the table passed to argument `fmt`.
3239

33-
34-
3540
### Access data
41+
3642
These properties stored in {class}`System <dpdata.System>` and {class}`LabeledSystem <dpdata.LabeledSystem>` can be accessed by operator `[]` with the key of the property supplied, for example
43+
3744
```python
3845
coords = d_outcar["coords"]
3946
```
40-
Available properties are (nframe: number of frames in the system, natoms: total number of atoms in the system)
4147

42-
| key | type | dimension | are labels | description
43-
| --- | --- | --- | --- | ---
44-
| 'atom_names' | list of str | ntypes | False | The name of each atom type
45-
| 'atom_numbs' | list of int | ntypes | False | The number of atoms of each atom type
46-
| 'atom_types' | np.ndarray | natoms | False | Array assigning type to each atom
47-
| 'cells' | np.ndarray | nframes x 3 x 3 | False | The cell tensor of each frame
48-
| 'coords' | np.ndarray | nframes x natoms x 3 | False | The atom coordinates
49-
| 'energies' | np.ndarray | nframes | True | The frame energies
50-
| 'forces' | np.ndarray | nframes x natoms x 3 | True | The atom forces
51-
| 'virials' | np.ndarray | nframes x 3 x 3 | True | The virial tensor of each frame
48+
Available properties are (nframe: number of frames in the system, natoms: total number of atoms in the system)
5249

50+
| key | type | dimension | are labels | description |
51+
| ------------ | ----------- | -------------------- | ---------- | ------------------------------------- |
52+
| 'atom_names' | list of str | ntypes | False | The name of each atom type |
53+
| 'atom_numbs' | list of int | ntypes | False | The number of atoms of each atom type |
54+
| 'atom_types' | np.ndarray | natoms | False | Array assigning type to each atom |
55+
| 'cells' | np.ndarray | nframes x 3 x 3 | False | The cell tensor of each frame |
56+
| 'coords' | np.ndarray | nframes x natoms x 3 | False | The atom coordinates |
57+
| 'energies' | np.ndarray | nframes | True | The frame energies |
58+
| 'forces' | np.ndarray | nframes x natoms x 3 | True | The atom forces |
59+
| 'virials' | np.ndarray | nframes x 3 x 3 | True | The virial tensor of each frame |
5360

5461
### Dump data
62+
5563
The data stored in {class}`System <dpdata.System>` or {class}`LabeledSystem <dpdata.LabeledSystem>` can be dumped in 'lammps/lmp' or 'vasp/poscar' format, for example:
64+
5665
```python
5766
d_outcar.to("lammps/lmp", "conf.lmp", frame_idx=0)
5867
```
68+
5969
The first frames of `d_outcar` will be dumped to 'conf.lmp'
70+
6071
```python
6172
d_outcar.to("vasp/poscar", "POSCAR", frame_idx=-1)
6273
```
74+
6375
The last frames of `d_outcar` will be dumped to 'POSCAR'.
6476

6577
The data stored in `LabeledSystem` can be dumped to deepmd-kit raw format, for example
78+
6679
```python
6780
d_outcar.to("deepmd/raw", "dpmd_raw")
6881
```
82+
6983
Or a simpler command:
84+
7085
```python
7186
dpdata.LabeledSystem("OUTCAR").to("deepmd/raw", "dpmd_raw")
7287
```
88+
7389
Frame selection can be implemented by
90+
7491
```python
7592
dpdata.LabeledSystem("OUTCAR").sub_system([0, -1]).to("deepmd/raw", "dpmd_raw")
7693
```
77-
by which only the first and last frames are dumped to `dpmd_raw`.
7894

95+
by which only the first and last frames are dumped to `dpmd_raw`.
7996

8097
### replicate
98+
8199
dpdata will create a super cell of the current atom configuration.
100+
82101
```python
83102
dpdata.System("./POSCAR").replicate(
84103
(
@@ -88,11 +107,13 @@ dpdata.System("./POSCAR").replicate(
88107
)
89108
)
90109
```
91-
tuple(1,2,3) means don't copy atom configuration in x direction, make 2 copys in y direction, make 3 copys in z direction.
92110

111+
tuple(1,2,3) means don't copy atom configuration in x direction, make 2 copys in y direction, make 3 copys in z direction.
93112

94113
### perturb
114+
95115
By the following example, each frame of the original system (`dpdata.System('./POSCAR')`) is perturbed to generate three new frames. For each frame, the cell is perturbed by 5% and the atom positions are perturbed by 0.6 Angstrom. `atom_pert_style` indicates that the perturbation to the atom positions is subject to normal distribution. Other available options to `atom_pert_style` are`uniform` (uniform in a ball), and `const` (uniform on a sphere).
116+
96117
```python
97118
perturbed_system = dpdata.System("./POSCAR").perturb(
98119
pert_num=3,
@@ -104,7 +125,9 @@ print(perturbed_system.data)
104125
```
105126

106127
### replace
128+
107129
By the following example, Random 8 Hf atoms in the system will be replaced by Zr atoms with the atom postion unchanged.
130+
108131
```python
109132
s = dpdata.System("tests/poscars/POSCAR.P42nmc", fmt="vasp/poscar")
110133
s.replace("Hf", "Zr", 8)

0 commit comments

Comments
 (0)