Developer Manual
Clone
GitHub
$ git clone git@github.com:lanl/waves.git
LANL
$ git clone ssh://git@re-git.lanl.gov:10022/aea/python-projects/waves.git
Dependencies
The development environment dependencies are found in the project’s environment.yml
file. For convenience, the file
is reproduced here
1channels:
2 - conda-forge
3
4dependencies:
5 - boa
6 # TODO: Remove when conda-build 24.? no longer breaks boa/mambabuild
7 - conda-build ==3.28
8 - h5netcdf
9 - h5py
10 - ipython
11 - matplotlib-base
12 - networkx
13 - numpy
14 - pandas
15 - pip
16 - pycodestyle
17 - pytest
18 - pytest-cov
19 - pytest-xdist
20 - python-build
21 - pyyaml
22 - salib >=1
23 - scipy >=1
24 - scons >=4
25 - seaborn
26 - setuptools >=62
27 - setuptools_scm[toml] >=6.2,<8
28 - sphinx >=5.1
29 - sphinx-argparse
30 - sphinx-copybutton >=0.5.1
31 - sphinx_rtd_theme >=1
32 - sphinxcontrib-bibtex
33 - xarray
In addition to the Conda environment managed packages, this project also requires a separately managed installation of TeXLive to build the PDF documentation.
The tutorials in this project may also require:
Abaqus
Cubit
Matlab
Activate Environment
Local development environments
SCons can be installed in a Conda environment with the Conda package manager. See the Conda installation and Conda environment management documentation for more details about using Conda.
Create the environment if it doesn’t exist
$ pwd path/to/local/git/clone/waves $ conda env create --name waves-env --file environment.yml
Activate the environment
$ conda activate waves-env
AEA CI server environment
A minimal development environment for the waves project Gitlab-CI jobs is maintained on AEA servers.
Add the AEA modulefiles directory
$ module use /projects/aea_compute/modulefiles
Load the project specific modulefile
$ module load waves-env
The Conda packages found in environment.yml
are reproduced in the Dependencies section.
Contribution Guidelines
The WAVES project uses the Gitlab Flow [25] workflow model.
Git Commit Message
Begin Git commit messages with one of the following headings:
BUG: bug fix
DOC: documentation
FEAT: feature
MAINT: maintenance
TST: tests
REL: release
WIP: work-in-progress
For example:
git commit -m "FEAT: short intent of new feature"
git commit -m "BUG: fixes nasty bug"
git commit -m "DOC: adds documentation for feature"
reStructured Text
Sphinx reads in docstrings and other special portions of the code as reStructured text. Developers should follow styles in this Sphinx style guide.
Style Guide
This project does not yet have a full style guide. Generally, wherever a style can’t be inferred from surrounding code this project falls back to PEP-8-like styles. There are two notable exceptions to the notional PEP-8 fall back:
Build
To build the Conda package activate the development environment and run the conda build command found in the CI configuration file. The current command may be found as
$ sed -n '/output_folder=/,/VERSION/p' .gitlab-ci.yml
...
$ output_folder='conda-bld'
$ mkdir ${output_folder}
$ VERSION=$(python -m setuptools_scm) conda build recipe --channel conda-forge --no-anaconda-upload --croot /scratch/${USER}/conda-build --output-folder ${output_folder}
A second recipe that bundles the LANL internally linked documentation is found in waves/recipe-internal
and can be
built similarly by replacing recipe
with recipe-internal
in the above command.
This project uses the SCons build system. This section will discuss some common build operations. For a full list of
SCons command line options and target build behavior, see the SCons manpage. The SCons manpage is also
installed with Scons in the environment and can be opened from the command line as man scons
in the AEA Compute
environment. In local environments, the manpage may not be in the MANPATH
. You can find the manpage file and
make them available with something similar to any of the following, in increasing order of required background
knowledge.
# Activate the environment
$ conda activate waves-env
# Find the scons manpage file
$ find $CONDA_PREFIX -name scons.1
/path/to/waves-env/scons.1
# Open manpage directly
$ man $CONDA_PREFIX/scons.1
# Link SCons manpage to expected path and update MANPATH
$ ln -s $CONDA_PREFIX/scons.1 $CONDA_PREFIX/man/man1/scons.1
$ export MANPATH=$MANPATH:$CONDA_PREFIX/man
$ man scons
This project contains several, separate SCons project configurations, where the SConstruct
file name indicates an
SCons project by convention. The WAVES package and documentation are defined in the waves/SConstruct
file. The
modsim template and regression tests are defined in a separate waves/modsim_template/Sconstruct
file. The WAVES
tutorials each have a tutorial specific configuration file waves/tutorials/*SConstruct
. The following build commands
apply to each, but must be run from their respective project configuration parent directories, waves
,
waves/modsim_template
and waves/tutorials
. The available targets and aliases differ accordingly.
When executing the tutorials or modsim template build commands directly in the repository, the WAVES project root
repository must be put on PYTHONPATH
. In personal (but not shared) virtual environments, the preferred method is
to run conda develop .
once from the project root directory. See the Conda documentation for more information
about “development” mode installs. For shared environments, the preferred solution is to prefix the following commands
with PYTHONPATH=.. ``, where it is assumed that the ``PWD
is the tutorial or modsim template root directory.
View project specific command line options, default targets, and aliases
$ scons -h ...
Build all default targets
$ scons
Build a specific target
$ scons <target name>
Remove the default targets’ artifacts
$ scons --clean
Remove all targets’ artifacts
$ scons . --clean
Test
WAVES has dedicated target aliases for the unit and system tests. To run the unit tests, activate a conda environment and run
$ scons pytest
The tutorials and modsim template are run as system tests and require third-party software not available on conda-forge.
To run the system tests, install the third-party software and make them available in your PATH
, activate a conda
environment and run
$ scons systemtest
The full list of continuous integration test commands can be found in the Gitlab-CI file, .gitlab-ci.yml
.
$ pwd
path/to/local/git/clone/waves/
$ sed -n '/fast-test/,/tags/p' .gitlab-ci.yml
The WAVES unit and system tests may also be executed from the installation directory. These tests use pytest directly
and may require non-default pytest options and markers to execute the system tests serially. The full list of CI
commands may be found in the internal recipe file, recipe-internal/meta.yaml
, under the test
keyword. The
following grep command will show the pytest commands and options.
$ grep "pytest " recipe-internal/meta.yaml
Test Local Module
When testing CLI changes locally, the waves module must be run as a script. We must also set the PYTHONPATH
in order to include the current waves module when operating on a configuration that imports waves.
Below is an example of a visualization test of an SConstruct file using the local waves module.
$ pwd
path/to/local/git/clone/waves/
$ PYTHONPATH=$PWD python -m waves.main visualize . --sconstruct /path/to/local/SConstruct
Documentation
The documentation build is automated with SCons as the documentation
target.
Build the WAVES documentation
$ pwd path/to/local/git/clone/waves/ $ scons documentation
Build the MODSIM-TEMPLATE documentation. The WAVES package must be on
PYTHONPATH
. For developers, the least disruptive solution is a per-command modification ofPYTHONPATH
.$ pwd path/to/local/git/clone/waves/modsim_template $ PYTHONPATH=/path/to/local/git/clone/waves:$PYTHONPATH scons documentation
Windows users
This project uses symbolic links to minimize duplication of files where possible. Some files, such as the tutorial and modsim template package files, can not be shared in common due to their applications and the necessary directory structure for each. However, if the file content is identical, a symbolic link is used to avoid duplicating the entire file.
Symbolic links require special handling on Windows computers. If contributors are developing from a Windows machine, they are encouraged to
Read about
mklink
and “developer mode” for Windows 10/11Use an up-to-date version of git
Use one of the following git configurations
# Global configuration. Run from anywhere. > git config --global core.symlinks true # Local configuration. Run from repository root directory after cloning. > git config core.symlinks true
Use unix line endings with one of the following git configurations
# Global configuration. Run from anywhere. > git config --global core.autocrlf true # Local configuration. Run from repository root directory after cloning. > git config core.autocrlf true