Over the years, I have explored how to scale up easily computation through Jupyter Notebooks by executing them not-interactively, possibily parametrized and remotely. This is mostly for reference.
nbsubmitis a Python package which has Python API to send a local notebook for execution on a remote SLURM cluster, for example Comet, see an example. This project is not maintained right now.- Back in 2017 I tested submitting notebooks to Open Science Grid, see the
batch-notebooks-condorrepository - Back in 2016 I created scripts to template a Jupyter Notebook and launch SLURM jobs, see
slurm.shared.templateandrunipyloop.sh