An accessible infrastructure for artificial intelligence using a Docker-based JupyterLab in Galaxy
Abstract: Background
Artificial intelligence (AI) programs that train on large datasets require powerful compute infrastructure consisting of several CPU cores and GPUs. JupyterLab provides an excellent framework for developing AI programs, but it needs to be hosted on such an infrastructure to enable faster training of AI programs using parallel computing.
Findings
An open-source, docker-based, and GPU-enabled JupyterLab infrastructure is developed that runs on the public compute infrastructure of Galaxy Europe consisting of thousands of CPU cores, many GPUs, and several petabytes of storage to rapidly prototype and develop end-to-end AI projects. Using a JupyterLab notebook, long-running AI model training programs can also be executed remotely to create trained models, represented in open neural network exchange (ONNX) format, and other output datasets in Galaxy. Other features include Git integration for version control, the option of creating and executing pipelines of notebooks, and multiple dashboards and packages for monitoring compute resources and visualization, respectively.
Conclusions
These features make JupyterLab in Galaxy Europe highly suitable for creating and managing AI projects. A recent scientific publication that predicts infected regions in COVID-19 computed tomography scan images is reproduced using various features of JupyterLab on Galaxy Europe. In addition, ColabFold, a faster implementation of AlphaFold2, is accessed in JupyterLab to predict the 3-dimensional structure of protein sequences. JupyterLab is accessible in 2 ways—one as an interactive Galaxy tool and the other by running the underlying Docker container. In both ways, long-running training can be executed on Galaxy’s compute infrastructure. Scripts to create the Docker container are available under MIT license at https://github.com/usegalaxy-eu/gpu-jupyterlab-docker
- Location
-
Deutsche Nationalbibliothek Frankfurt am Main
- Extent
-
Online-Ressource
- Language
-
Englisch
- Notes
-
GigaScience. - 12 (2022) , giad028, ISSN: 2047-217X
- Event
-
Veröffentlichung
- (where)
-
Freiburg
- (who)
-
Universität
- (when)
-
2023
- Creator
- DOI
-
10.1093/gigascience/giad028
- URN
-
urn:nbn:de:bsz:25-freidok-2368578
- Rights
-
Open Access; Der Zugriff auf das Objekt ist unbeschränkt möglich.
- Last update
-
25.03.2025, 1:52 PM CET
Data provider
Deutsche Nationalbibliothek. If you have any questions about the object, please contact the data provider.
Associated
- Kumar, Anup
- Cuccuru, Gianmauro
- Grüning, Björn
- Backofen, Rolf
- Universität
Time of origin
- 2023
Other Objects (12)
Expressivity and decidability of first-order languages over feature trees
Expressivity and decidability of first-order languages over feature trees
Transformer-based tool recommendation system in Galaxy
A complete axiomatization of a theory with feature and arity constraints
Regular path expressions in feature logic
Tool recommender system in Galaxy using deep learning
Workflow recommendations using deep learning and machine learning tools for exploratory and predictive data analysis in Galaxy
Optimizing algorithms for the comparative analysis of non-coding RNAs
INFO-RNA—a server for fast inverse RNA folding satisfying sequence constraints
Computational analyses of post-transcriptional regulatory mechanisms : = Computergestützte Analysen von post-transkriptionellen regulatorischen Mechanismen
A complete axiomatization of a theory with feature and arity constraints
Using local RNA secondary structures for computational comparison and clustering of RNA molecules
Expressivity and decidability of first-order languages over feature trees
Expressivity and decidability of first-order languages over feature trees
Transformer-based tool recommendation system in Galaxy
A complete axiomatization of a theory with feature and arity constraints
Regular path expressions in feature logic
Tool recommender system in Galaxy using deep learning
Workflow recommendations using deep learning and machine learning tools for exploratory and predictive data analysis in Galaxy
Optimizing algorithms for the comparative analysis of non-coding RNAs
INFO-RNA—a server for fast inverse RNA folding satisfying sequence constraints
Computational analyses of post-transcriptional regulatory mechanisms : = Computergestützte Analysen von post-transkriptionellen regulatorischen Mechanismen
A complete axiomatization of a theory with feature and arity constraints
Using local RNA secondary structures for computational comparison and clustering of RNA molecules
Expressivity and decidability of first-order languages over feature trees
Expressivity and decidability of first-order languages over feature trees
Transformer-based tool recommendation system in Galaxy
A complete axiomatization of a theory with feature and arity constraints
Regular path expressions in feature logic
Tool recommender system in Galaxy using deep learning
Workflow recommendations using deep learning and machine learning tools for exploratory and predictive data analysis in Galaxy
Optimizing algorithms for the comparative analysis of non-coding RNAs
INFO-RNA—a server for fast inverse RNA folding satisfying sequence constraints
Computational analyses of post-transcriptional regulatory mechanisms : = Computergestützte Analysen von post-transkriptionellen regulatorischen Mechanismen
A complete axiomatization of a theory with feature and arity constraints