Add script to retrieve data

README.md

@@ -18,10 +18,25 @@ This repository includes all the scripts written to produce figures in the paper
 
 # Usage
 
+## Retrieve citations
+
 - Launch [Jupyter](https://jupyter.org/) to access the notebooks to generate graphs
 
     ```
     $ make run-jupyter
     ```
 
 - Go to [http://localhost:8888](http://localhost:8888) (a page should open automatically in your browser)
+
+## Update data
+
+- Launch the dedicated script
+
+    ```
+    $ bash bin/get_data.sh
+    ```
+
+
+
+
+

bin/get_data.sh

@@ -0,0 +1,48 @@
+#!/usr/bin/env bash
+
+# Get platform xlsx
+wget \
+    -O docs/supplementary/supplementary_table_1.xlsx \
+    "https://docs.google.com/spreadsheets/d/1KxWoVRvuusMiKvs0ELiKHjbbQBBshdntSW1zxB4jyXc/export?format=xlsx"
+
+# Get survey data
+wget \
+    -O "docs/supplementary/supplementary_table_2.tsv" \
+    "https://raw.githubusercontent.com/paulzierep/microgalaxy-survey/refs/heads/main/data/modified_answers.csv" 
+
+
+# Get Codex data
+wget \
+    -O "docs/supplementary/supplementary_table_3.tsv" \
+    "https://raw.githubusercontent.com/galaxyproject/galaxy_codex/refs/heads/main/communities/microgalaxy/resources/curated_tools.tsv" 
+
+
+wget \
+    -O "docs/supplementary/supplementary_table_4.tsv" \
+    "https://raw.githubusercontent.com/galaxyproject/galaxy_codex/refs/heads/main/communities/microgalaxy/resources/curated_workflows.tsv" 
+
+
+wget \
+    -O "docs/supplementary/supplementary_table_5.tsv" \
+    "https://raw.githubusercontent.com/galaxyproject/galaxy_codex/refs/heads/main/communities/microgalaxy/resources/tutorials.tsv" 
+
+
+# Get training events
+wget \
+    -O "docs/supplementary/supplementary_table_6.tsv" \
+    "https://docs.google.com/spreadsheets/d/1Eejqmb4CJQnyy51ayLMOgTPf5mHArKlCtSYzEsFNc2s/export?format=tsv&gid=0" 
+
+
+# Get use cases
+wget \
+    -O "docs/supplementary/supplementary_table_7.tsv" \
+    "https://docs.google.com/spreadsheets/d/1fGbDFk5u4vwrjcdN8zWXHEvxkLUYJSx_nbK1XUeSZyk/export?format=tsv&gid=1847337048" 
+
+# Get Supplementary documents
+wget \
+    -O "docs/supplementary/supplementary_document_1.pdf" \
+    "http://docs.google.com/document/d/1MuR1c8NbVydhQG_CoyLzIjd4KYbECipl15t5DJk6vKI/export?format=pdf"
+
+wget \
+    -O "docs/supplementary/supplementary_document_2.pdf" \
+    "http://docs.google.com/document/d/16-P_NCOfFo0gezU51aev5OBuutiiR8DPz9KrZ7JSgmk/export?format=pdf"

docs/supplementary/supplementary_table_7.tsv

@@ -0,0 +1,22 @@
+Title	Status	Publication	Keywords	Main target	Methods	Data generation	Data analysis	Specific benefits of using Galaxy	Section in the manuscript
+A user friendly and high-throughput way to identify and analyse SNPs in Plasmodium falciparum (malaria parasites)  	On-going	No	Amoeba, Unicellular eukaryotes, Malaria, Userfriendly, sequence diversity, high-throughput	SNP analysis (molecular markers of antimalarial drug resistance)	Targeted amplicon sequencing	Targeted amplicon sequencing on an Illumina miseq platform, however, we have our own barcoding system that exceeds the possibilities in Iluminas own barcoding. 	Quality control, trimming, assembling, alignment to reference, SNP analysis, statistical analysis	Online servers for the processing of data Access to the tools Easy to use tools	
+Staphylococcus aureus Intensive server for Research In Omics data	On-going	Yes	Officila Laboratory food analysis, genomic bacterial characterizatio, foodbourne outbreak resolution	Gene identification, SNP analysis, genome annotation	Single organism genomics	Illumina	Quality control, assembly, genome annotation, enterotoxins gene detection, cgMLST	Galaxy allows users with no specific bioinformatics skills or dedicated hardware to conduct computational intensive data analysis on bacterial genomes. Moreover, the possibility to share workflows with users, automating the analysis of raw sequencing data from the initial quality control of the sequences to the assembly and annotation of the genomes, and finally to perform phylogenetic inference in the event of microbial outbreaks. 	
+Investigation of the Defective Growth Pattern and Multidrug Resistance in a Clinical Isolate of Candida glabrata Using Whole-Genome Sequencing and Computational Biology Applications	Completed	Yes	Microbiology, Antifungal drug resistance, Candida glabrata, Mycology	Pathogen, Antimicrobial drug resistance, variant analysis, variant annotation, genome assembly, single pathogen genomics	Single organism genomics	Illumina Nextseq 550	Quality control (FASTQC) and trimming (Trimmomatic)Alignment (Bowtie2)Variant calling (Lofreq suite)Variant annotation (SnpEff)Genome assembly (Spades)Genome annotation (Funannotate)	access to computational power vital for microbial organisms with larger genomes  access to databases tool availability easy-to-use GUI reproducibility community Being able to continue your daily tasks while your data has been processed Being able to close the browser and return when your analysis is done Being able to check the process and run tools with a smartphone browser Being able to access your data and results at any time, anywhere Knowing your data is secure Being able to share history with colleagues and partners 	
+Multiomics studies on novel trichoderma strains	On-going	Yes	Whole genome analysis, genomics, multiomics	Explore novel Trichoderma strain functional potential and on-going strain characterization	Single organism genomics, Transcriptomics, structural variation comparison between whole genomes, sinteny and colinearity, Kegg, BiologFF carbon phenotype array	Pacbio, Illumina, biologFF phenotype Array	Quality control, Differential expression analysis, genomic characterization	Easily accessible from multiple location, shared storage and analysis, user friendly for complex tasks	
+Evaluation of long read assemblers for ONT reads of prokaryotes and metagenomes	On-going	Under preparation	Nanopore sequencing, long reads, prokaryotes, metagenomes, benchmarking	Comparative analysis, tool evaluation	metagenomic	Nanopore	Quality control, Assembly, Polishing, Statistical analysis	Access computational power, tool availability	
+Free living amoeba as vectors for transmission of emerging pathogens and antibiotic resistance genes in biological wastewater treatment plants 	Completed	Submitted	Wastewater, Metagenomics, ARB, ARG, Free living amoeba 	Microbiome community, Gene identification, AMR and pathogens	Shotgun metagenomics	Illumina	Quality control, taxonomy profiling and functional gene abundance	Easy to use, computational power, tool availability, workflow access, technical support and  help during analysis	
+Zooplankton influences extracellular DNA degradation and acquisition by natural transformation in freshwater microcosms through the secretion of different enzymes	On-going	Under preparation	Daphnia; microbiome; secretome; LC-MS proteomics; DNA transformation	Microbiome community; AMR; secretome.	Proteomics	LC-MS/MS	We used Galaxy to build the protein database for the identification of peptides from zooplankton (i.e.Daphnia) and its microbiome, based on previous identification by our group (Eckert et al., 2016). Galaxy was mainly used for the identification of the proteins secreted. We then performed functional analysis of proteins by using external tools. 	- Access to computational power; - reduced local disk space occupation; - sharing with colleagues; - reproducibility; - easy to compare different tools.	
+Metaproteomic Analysis of Nasopharyngeal Swab Samples to Identify Microbial Peptides in COVID-19 Patients.	Completed	Yes	parallel reaction monitoring, COVID-19 pandemic, metaproteomics	Pathogen, Co-infection	Metaproteomics; Mass Spectrometry; Proteomics	LC-MS/MS	Protein Sequences Database Generation for Microbial Peptide Detection and VerificationDiscovery Workflow; Microbial Peptide Verification Workflow Using PepQuery; Taxonomic and Functional Analysis; Quantitative and Statistical Analysis of Microbial Proteins.	Computational power; Shareability of workflows and histories; Encourages collaborations; Accessibility to tools; Reproducibility.	
+In vivo genotypic evolution of Klebsiella pneumoniae Sequence Type 512 under ceftazidime-avibactam, meropenem-vaborbactam and cefiderocol treatment	Completed	Yes	inhibitor-resistant KPC, colistin resistance, carbapenemase co-production, outer membrane porin disruption, siderophore disruption	AMR, Klebsiella pneumoniae, phylogenomics	Genomics	Illumina, Nanopore	Assembly, phylogenomics, protein structure prediction	access to computational power, reproducibility and bulk processing using collections	
+Phenotypic microarray and molecular identification of plant growth promoting rhizobacteria of selected bioenergy plants	Completed	Yes	Non-edible oil seed plants, Metagenomics, CLPP, Phenotypic microarray, Plant Microbe Interaction 	Microbiome community, Plant growth promoting bacteria Identification 	16S rRNA amplicon sequencing	Illumina	Quality control, taxonomy profiling and functional gene abundance	Easy to use, computational power, tool availability, workflow access, technical support and  help during analysis	
+Effect of bioinoculants on the structural and functional dynamics of soil microbiome 	On-going	Under preparation	 Microalgal bioinoculants, Capsicum annuum, Growth, Microbiome community, and functional genes abundance	Taxonomic changes, function analysis, gene identification  and pathway prediction 	Shotgun metagenomics 	Illumina	Quality control, taxonomy profiling, functional genes abundances and pathway analysis 	Easy to use, computational power, tool availability, workflow access, technical support and  help during analysis	
+To analyse the long term effect of sulphide oj Annamox process 	On-going	Under preparation	Anammox, Biological wastewater treatment, Sulphide dosing, Microbiome  	Microbial community profiling, diversity analysis, Functional analysis and gene identification  	Shotgun metagenomics 	Illumina	Quality control, taxonomy profiling, functional genes abundances and pathway analysis	Easy to use, computational power, tool availability, workflow access, technical support and  help during analysis	
+Integrative meta-omics in Galaxy and beyond	Completed	Yes		Microbiome community, MAGs building, Functional analysis,  multiomics	Metagenomics, Metatranscriptomics, Metaproteomics	Illumina, nanoLC-MS/MS	3 separate workflows for (1) metagenomic assembly, binning, and functional annotation (MetaG), (2) metatranscriptomics (MetaT), and (3) metaproteomics (MetaP)	easy-to-use workflows (nesting of programs together is way more difficult on HPCs e.g., using Snakemake)	
+Clouds influence the functioning of airborne microorganisms	Completed	BioRxiv preprint, in submission to BioGeosciences	Clouds, microbiome, metagenomics, metatranscriptomics	Microbiome community, Functional analysis, Gene identification	Shotgun metagenomics, Metatranscriptomics	Illumina	Quality control, Assembly, Taxonomy profiling, Functional annotation, Construction of a gene catalog, Differential expression analysis	Galaxy instance deployed by the bioinformatics facility hosted by the University (Auvergne BioInformatique, AuBI), and the regional calculation cluster Mesocentre Clermont Auvergne	
+ASaiM-MT: a validated and optimized ASaiM workflow for metatranscriptomics analysis within Galaxy framework	Completed	Yes	Galaxy, metatranscriptomics, microbiome, functional analysis	Metatranscriptomics data analysis, Galaxy Platform, taxonomy annotation, functional annotation	Metatranscriptomics		Galaxy Platform, Quality control, metatranscriptome assembly, taxonomic profiling, functional profiling, taxonomy and functional relationship	Easy-to-use workflows, handling large datasets and databases, easy for workflow development, most open source tools are available, access to computational power, reproducibility, and shareability.	
+Heli Project: The intestinal microbiome as the centrepiece of the human body’s reaction to colonization and infection with Helicobacter pylori	On-going		Human health; Helicobacter; meta-transcriptome; immune defence; metabolism; machine learning	Microbiome community, Gene identification, Functional analysis, SNP analysis, Machine learning	Metagenomics, metatranscriptomics, amplicon, transcriptomics, metabolomics	Illumina, MS	Quality control, Taxonomy/community profiling, Functional profiling, SNP and variant calling, Machine learning//Representation learning	easy-to-use workflows, bulk processing using collections, computational power, reproducibility, adaptability	
+Accessible and scalable detection and identification of foodborne pathogens	Completed	In submission	Quality controlling, Contamination/Host removal, Taxonomy profiling, Assembly, Pathogens detection, AMR identification, VF gene identification, MLST, SNP analysis, FAIR Galaxy workflows, GTN material, visualization, reports, automation	Foodborne pathogens, microbiome community, Gene identification, SNP analysis, AMR, Virulence Factor, 	Metagenomics	Nanopore	5 separate workflows: (1) Quality control and Contamination/Host Removal; (2) Taxonomy Profiling; (3) Assembly, Virulence Factor and AMR Genes identification; (4) SNP and Variant calling; (5) Statistical analysis and Visualisation	Within the project we needed to create easy to use, findable, adaptable, reusable and accessible workflows. With Galaxy, we found all the tools we needed to create these workflows. We also found the flexibility to add or update any tool we need, same for databases. Galaxy also has the power to return the results along with the visualisations and reports all in one place (history). So users of our Galaxy created workflows, will not only be able to run the workflows easily and get the results directly, but also can adapt the workflows to any other type of sequencing technique or any other analysis and that is because Galaxy allows users to easily do that. Furthermore, using Galaxy bot, the whole process of analysing and tracking pathogens within samples can be automated, to run and return results directly after a new sample is uploaded. Galaxy’s bulk processing and computational power allows users to run the workflows with collections of huge datasets.	
+What is in the microbiome of beers? Aggregation of public beer microbiome data, development and evaluation of FAIR beer microbiome workflows, and implementation of BeerMicroDB, a comprehensive database for beer microbiome	On-going		Beer microbiome, public data	Microbiome community	Metabarcoding, Metagenomics	Illumina, Nanopore	Quality control, Taxonomy profiling, Diversity analysis	User-friendly interface, simplified construction of workflows Robust support system, i.e. community Bioblend, a Python library for automating extensive data analyses, and simplifying the management of large and complex bioinformatics projects	
+DNAnalyzer: an online & interactive educational game on microbiome DNA data analysis and research	On-going		Game, Education, Data analysis, Research practice	Microbiome community	Metagenomics	Nanopore	Quality control, Taxonomy profiling	Access to computational power Tool availability	
+SARS-CoV-2 detection in wastewater	On-going								
+Feature selection with vector-symbolic architectures: a case study on microbial profiles of shotgun metagenomic samples of colorectal cancer	Completed	Under review on GigaScience	Metagenomics, machine learning, feature selection, colorectal cancer, microbial biomarkers, hyperdimensional computing, vector-symbolic architecturesq	Microbial biomarkers	Metagenomics	Illumina	Taxonomic profiling, statistical analysis, samples classification, feature selection	Access to computational power; reproducibility