Monday, 2 July 2018

SLiMSuite Downloads

The current SLiMSuite release is v1.4.0 (2018-07-02) and can be downloaded by clicking the button (left).

In addition to the tarball available via the links above, SLiMSuite is available as a GitHub repository (right).


See also: Installation and Setup.

Previous Releases

SLiMSuite release v1.4.0 (2018-06-02) now oline

SLiMSuite release v1.4.0 (2018-07-02) is now on GitHub. The REST servers have also been updated to run this version of the code.

This release of SLiMSuite contains a number of updates related to the REST servers and some new pre-release dev tools in the main repo (but not the *.tgz file).

SeqList has updated sequence summary statistics and grep-based redundancy removal for large genomes.

One major bug fix is a change to parsing Uniprot entries from the website following a change in behaviour of the API.

SLiMSuite updates

Updates in extras/:

• rje_pydocs: Updated from Version 2.16.3.
→ Version 2.16.4: Tweaked formatDocString.
→ Version 2.16.5: Added general commands to docstring HTML for REST servers.
→ Version 2.16.6: Modified parsing to keep DocString for SPyDarm runs.
→ Version 2.16.7: Fixed T/F/FILE option type parsing bug.

Updates in libraries/:

• rje_blast_V2: Updated from Version 2.18.0.
→ Version 2.19.0: Added blastgz=T/F : Whether to zip and unzip BLAST results files [False]
→ Version 2.19.1: Fixed erroneous i=-1 blastprog over-ride but not sure why it was happening.
→ Version 2.20.0: Added localGFF output
→ Version 2.21.0: Added blasttask=X setting for BLAST -task ['megablast']
→ Version 2.22.0: Added dust filter for blastn and setting blastprog based on blasttask
→ Version 2.22.1: Added trimLocal error catching for exonerate issues.
→ Version 2.22.2: Fixed GFF attribute case issue.

• rje_db: Updated from Version 1.8.6.
→ Version 1.9.0: Added comment output to saveToFile().

• rje_disorder: Updated from Version 0.8.
→ Version 1.0.0: Added random disorder function and elevated to v1.x as fully functional for SLiMSuite
→ Version 1.1.0: Added strict option for disorder method selection. Added minorder=X.
→ Version 1.2.0: Added saving and loading scores to IUScoreDir/.

• rje_gff: Created/Renamed/moved.
→ Version 0.0.0: Initial Compilation.
→ Version 0.1.0: Basic functional version.

• rje_hpc: Updated from Version 1.1.
→ Version 1.1.1: Added output of subjob command to log as run.

• rje_html: Updated from Version 0.2.1.
→ Version 0.3.0: Added optional loading of javascript files and stupidtable.js?dev default.

• rje_qsub: Updated from Version 1.9.1.
→ Version 1.9.2: Modified qsub() to return job ID.

• rje_samtools: Updated from Version 1.19.2.
→ Version 1.20.0: Added parsing of BAM file - needs samtools on system. Added minsoftclip=X, maxsoftclip=X and minreadlen=X.

• rje_seq: Updated from Version 3.24.0.
→ Version 3.25.0: 9spec=T/F : Whether to treat 9XXXX species codes as actual species (generally higher taxa) [False]

• rje_seqlist: Updated from Version 1.25.0.
→ Version 1.26.0: Updated sequence statistics and fixed N50 underestimation bug.
→ Version 1.26.1: Fixed median length overestimation bug.
→ Version 1.26.2: Fixed sizesort bug. (Now big to small as advertised.)
→ Version 1.27.0: Added grepNR() method (dev only). Switched default to RevCompNR=T.
→ Version 1.28.0: Fixed second pass NR naming bug and added option to switch off altogether.
→ Version 1.29.0: Added maker=T/F : Whether to extract MAKER2 statistics (AED, eAED, QI) from sequence names [False]

• rje_slimcalc: Updated from Version 0.9.3.
→ Version 0.10.0: Added extra disorder methods to slimcalc.

• rje_taxonomy: Updated from Version 1.2.0.
→ Version 1.3.0: taxtable=T/F : Whether to output results in a table rather than text lists [False]

• rje_tree: Updated from Version 2.15.0.
→ Version 2.16.0: 9spec=T/F : Whether to treat 9XXXX species codes as actual species (generally higher taxa) [False]
→ Version 2.16.1: Modified NSF reading to cope with extra information beyond the ";".

• rje_uniprot: Updated from Version 3.24.1.
→ Version 3.24.2: Updated HTTP to HTTPS. Having some download issues with server failures.
→ Version 3.25.0: Fixed new Uniprot batch query URL. Added onebyone=T/F : Whether to download one entry at a time. Slower but should maintain order [False].

• rje_zen: Updated from Version 1.3.2.
→ Version 1.4.0: Added some more words and "They fight crime!" structure.

Updates in tools/:

• gablam: Updated from Version 2.28.3.
→ Version 2.29.0: Added localGFF=T/F output

• gasp: Updated from Version 1.4.
→ Version 2.0.0: Upgraded to rje_obj framework for REST server.

• gasp_V1: Created/Renamed/moved.
→ Version 0.0: Initial Compilation.
→ Version 1.0: Improved version with second pass.
→ Version 1.1: Improved OO. Restriction to descendant AAs. (Good for BAD etc.)
→ Version 1.2: No Out Object in Objects
→ Version 1.3: Added more interactive load options
→ Version 1.4: Minor tweaks to imports.

• gopher: Updated from Version 3.4.2.
→ Version 3.4.3: Added checking and warning if no bootstraps for orthtree.

• haqesac: Updated from Version 1.11.0.
→ Version 1.12.0: 9spec=T/F : Whether to treat 9XXXX species codes as actual species (generally higher taxa) [False]

• multihaq: Updated from Version 1.3.0.
→ Version 1.4.0: Added SLiMFarmer batch forking if autoskip=F and i=-1.
→ Version 1.4.1: Added haqblastdir=PATH: Directory in which MultiHAQ BLAST2FAS BLAST runs will be performed [./HAQBLAST/]

• pagsat: Updated from Version 2.3.3.
→ Version 2.3.4: Fixed full.fas request bug.
→ Version 2.4.0: Added PAGSAT compile mode to generate comparisons of reference chromosomes across assemblies.

• seqsuite: Updated from Version 1.14.0.
→ Version 1.14.1: Added zentest for testing the REST servers.
→ Version 1.15.0: Added GASP to REST servers.
→ Version 1.16.0: Add rje_gff.GFF to REST servers.
→ Version 1.17.0: Added batch summarise mode.
→ Version 1.18.0: Added rje_apollo.Apollo to REST servers.
→ Version 1.19.0: Tweaked the output of batch summarise, adding Gap% and reducing dp for some fields.
→ Version 1.19.1: Fixed GapPC summarise output to be a percentage, not a fraction.

• slimbench: Updated from Version 2.14.0.
→ Version 2.14.1: Fixed up PPIBench results loading.
→ Version 2.14.2: Fixed ByCloud bug.
→ Version 2.15.0: Updated assessSearchMemSaver() to handle different data types properly. dombench not yet supported.
→ Version 2.16.0: Added ppi hub/slim summary and motif filter for assessment datasets post-rating (still count as OT)
→ Version 2.16.1: Bug-fixing PPI generation from pairwise PPI files.
→ Version 2.16.2: Fixed benchmarking setup bug.
→ Version 2.16.3: Fixed bug when Hub-PPI links fail during PPI Benchmarking.
→ Version 2.17.0: Added output of missing datasets when balanced=T.
→ Version 2.18.0: Added dev OccBench with improved ratings and more efficient results handling. (dev only)
→ Version 2.18.1: Added additional OccBench options (bymotif, occsource, occspec)
→ Version 2.18.2: Fixed problem with source file selection ignoring i=-1.

• slimfarmer: Updated from Version 1.7.0.
→ Version 1.8.0: jobforks=X : Number of forks to pass to farmed out run if >0 [0]
→ Version 1.9.0: daisychain=X : Chain together a set of qsub runs of the same call that depend on the previous job.

• slimfinder: Updated from Version 5.3.3.
→ Version 5.3.4: Fixed terminal (^/$) musthave bug.

• slimsuite: Updated from Version 1.7.0.
→ Version 1.7.1: Added error raising for protected REST alias data.

• smrtscape: Updated from Version 2.2.1.
→ Version 2.2.2: Added dna=T to all SeqList object generation.

• snapper: Updated from Version 1.6.0.
→ Version 1.6.1: Fixed bug for reducing to unique-unique pairings that was over-filtering.

© RJ Edwards 2018. Last modified 2 Jul 2018.

Tuesday, 16 January 2018

SLiMSuite REST server is back up

The REST server is back up. The development server is currently having an upgrade and should not be used.

SLiMSuite REST server is currently down

The SLiMSuite REST server is experiencing some technical difficulties at the moment. It will hopefully be back up soon.

Tuesday, 19 December 2017

SLiMSuite REST Servers updated

The SLiMSuite REST Servers have been updated to the latest release code (v1.3.0). Please report any issues!

SLiMSuite release v1.3.0 (2017-12-18) online

SLiMSuite release v1.3.0 (2017-12-18) is now on GitHub. Funding for SLiMSuite development is proving elusive at present, so this release is a little less organised (and later) than planned. The main additions are various programs in development for PacBio genomics and a draft SLiMSuite parser Shiny app in the new shiny/ directory. The old packages/ directory has also been removed. Check the docs/release/ files and see below for more information on this release.

Another release with improved documentation is currently planned for early 2018. As ever, if you want access to the latest code, email to download the full svn repository.

SLiMSuite updates

Updates in extras/:

• rje_dbase: Updated from Version 2.3.
→ Version 2.3.1: Updated the dbdownload function to recognise individual files and wildcard file lists.

• rje_pydocs: Updated from Version 2.16.3.
→ Version 2.17.4: Tweaked formatDocString.
→ Version 2.17.5: Added general commands to docstring HTML for REST servers.

Updates in legacy/:

Updates in libraries/:

• rje: Updated from Version 4.17.0.
→ Version 4.18.0: Added Roman numeral functions.
→ Version 4.18.1: Updated error handling for full REST output.
→ Version 4.18.2: Fixed rje module call bug.
→ Version 4.19.0: Tweaked Docstring. Added extra parameter catching. Added report of INI loading.

• rje_blast_V2: Updated from Version 2.11.2.
→ Version 2.12.0: Added localidcut %identity filter for GABLAM calculations.
→ Version 2.13.0: Added GFF and SAM output for BLAST local tables for GABLAM, PAGSAT etc.
→ Version 2.14.0: Updated gablamfrag=X and fragmerge=X usage. Fixed localFragFas position output.
→ Version 2.15.0: Fragmerge no longer removes flanks and can be negative for enforced overlap!
→ Version 2.16.0: Added qassemblefas mode for generating fasta file from outfmt 4 run.
→ Version 2.16.1: Improved error messages for BLAST QAssembly.
→ Version 2.17.0: qconsensus=X : Whether to convert QAssemble alignments to consensus sequences (None/Hit/Full) [None]
→ Version 2.17.1: Modified QAssembleFas output sequence names for better combining of hits. Added QFasDir.
→ Version 2.17.2: Modified QAssembleFas output file names for better re-running. Fixed major QConsensus Bug.
→ Version 2.18.0: Added REST output. Fixed QConsensus=Full bug.

• rje_db: Updated from Version 1.8.1.
→ Version 1.8.2: Fixed minor readSet bug.
→ Version 1.8.3: Minor debugging message changes.
→ Version 1.8.4: Cosmetic log message changes.
→ Version 1.8.5: Added saveToFileName() function.
→ Version 1.8.6: Minor IndexReport tweak.

• rje_genbank: Updated from Version 1.5.2.
→ Version 1.5.3: Fixed https genbank download issue.

• rje_menu: Updated from Version 0.4.0.
→ Version 0.5.0: Enabled simpler return tuples.

• rje_obj: Updated from Version 2.2.1.
→ Version 2.2.2: Updated error handling for full REST output.

• rje_qsub: Updated from Version 1.6.3.
→ Version 1.7.0: Added option for email when job started
→ Version 1.8.0: Added modpurge=T/F : Whether to purge loaded modules in qsub job file prior to loading [True]
→ Version 1.9.0: Added precall=LIST : List of additional commands to run between module loading and program call []
→ Version 1.9.1: Removed default module list: causing conflicts. Better to have in INI file.

• rje_samtools: Updated from Version 1.8.1.
→ Version 1.9.0: Added depthplot data generation. (Will need to add R function for plot itself.)
→ Version 1.9.1: Changed mincut default to 0.1.
→ Version 1.10.0: Added readlen output, which is like the depth plot but uses max read length (kb) instead of depth.
→ Version 1.11.0: Added dirnlen=X : Include directional read length data at X bp intervals (depthplot=T; 0=OFF) [500]
→ Version 1.11.1: Minor tweaks to try and speed up pileup parsing.
→ Version 1.12.0: Updated the snpfreq run code to make clearer and check for parsing issues. Set mincut=1 default.
→ Version 1.13.0: Added skiploci=LIST - need to screen out mitochondrion from Illumina Pileup parsing!
→ Version 1.14.0: Added forking of pileup parsing for SNPFreq analysis.
→ Version 1.14.1: Fixed SNPFreq rerunning bug.
→ Version 1.15.0: Added rgraphics=T/F : Whether to generate snpfreq multichromosome plots [True]
→ Version 1.16.0: Add coverage calculation per locus to depth plot table output (depthplot=T).
→ Version 1.16.1: Added reporting of existing files for parsing Pileup.
→ Version 1.17.0: Added parsing of lengths from SAM files to RID file.
→ Version 1.18.0: Updated processing of Treatment and Control without Alt to still limit to SNPTable. Fixed SNPFreq filters.
→ Version 1.19.0: snptableout=T/F : Output filtered alleles to SNP Table [False]
→ Version 1.19.1: Fixed AltLocus SNP table bug.
→ Version 1.19.2: Updated forker parsing to hopefully fix bug.

• rje_seqlist: Updated from Version 1.20.1.
→ Version 1.21.0: Added capacity to add/update database object from self.summarise() even if not seqmode=db. Added filedb mode.
→ Version 1.22.0: Added geneDic() method.
→ Version 1.23.0: Added seqSequence() method.
→ Version 1.24.0: Add NNN gaps option and "delete rest of sequences" to edit().
→ Version 1.24.1: Minor edit bug fix and DNA toggle option.
→ Version 1.25.0: Added loading of FASTQ files in seqmode=file mode.

• rje_sequence: Updated from Version 2.5.3.
→ Version 2.6.0: Added mutation dictionary to Ks calculation.

• rje_slim: Updated from Version 1.12.0.
→ Version 1.12.1: Modified error message.

• rje_slimcalc: Updated from Version 0.9.2.
→ Version 0.9.3: Changed fudge error to warning.

• rje_slimcore: Updated from Version 2.7.7.
→ Version 2.7.8: Fixed batch=FILE error for single input files.
→ Version 2.8.0: Added map and failed output to REST servers and standalone uniprotid=LIST input runs.
→ Version 2.8.1: Updated resfile to be set by basefile if no resfile=X setting given
→ Version 2.9.0: Added separate IUPred long suffix for reusing predictions

• rje_synteny: Updated from Version 0.0.0.
→ Version 0.0.1: Altered problematic ValueError to warnLog()
→ Version 0.0.2: Updated the synteny mappings to be m::n instead of m:n for Excel compatibility.
→ Version 0.0.3: Added catching of the Feature locus/accnum mismatch issue.

• rje_tree: Updated from Version 2.14.0.
→ Version 2.14.1: Fixed clustalw2 makeTree issue.
→ Version 2.15.0: Added IQTree.

• rje_uniprot: Updated from Version 3.22.0.
→ Version 3.23.0: Added accnum map table output. Fixed REST output bug when bad IDs given. Added version and about output.
→ Version 3.24.0: Added pfam out and changed map table headers.
→ Version 3.24.1: Fixed process Uniprot error when uniprot=FILE given.

• rje_zen: Updated from Version 1.3.1.
→ Version 1.3.2: Added some more words.

• snp_mapper: Updated from Version 1.0.0.
→ Version 1.1.0: Added pNS and modified the "Positive" CDS rating to be pNS < 0.05.
→ Version 1.1.1: Updated pNS calculation to include EXT mutations and substitution frequency.
→ Version 1.2.0: SNPByFType=T/F : Whether to output mapped SNPs by feature type (before FTBest filtering) [False]

Updates in tools/:

• gablam: Updated from Version 2.23.0.
→ Version 2.23.1: Added tuplekeys=T to cmd_list as default. (Can still be over-ridden if it breaks things!)
→ Version 2.24.0: Added localidmin and and localidcut as %identity versions of localmin and localcut. (Use for PAGSAT.)
→ Version 2.25.0: Added localsAM=T/F : Save local (and unique) hits data as SAM files in addition to TDT [False]
→ Version 2.26.0: Fixed fragfas output and clarified fullblast=T/F, localmin=X and localcut=X. Set fullblast=T keepblast=T.
→ Version 2.26.1: Fixed keepblast error.
→ Version 2.26.2: Fixed gablamcut fragfas filtering bug.
→ Version 2.26.3: Fixed nrseq=T to use Query OR Hit stat for NR filtering.
→ Version 2.26.4: Minor bug fix to nrchoice command parsing.
→ Version 2.27.0: Fragmerge no longer removes flanks and can be negative for enforced overlap!
→ Version 2.28.0: Added localidmin=PERC to localUnique (and thus Snapper).
→ Version 2.28.1: Fixed missing combinedfas when using existing blastres.
→ Version 2.28.2: Minor bug fix for NRSeq manual choice when i=-1.
→ Version 2.28.3: Fixed NRSeq query sorting bug.

• haqesac: Updated from Version 1.10.2.
→ Version 1.10.3: Added catching of bad query when i=-1.
→ Version 1.11.0: Added resdir=PATH [./HAQESAC/] for d>0 outputs.

• multihaq: Updated from Version 1.2.2.
→ Version 1.3.0: MultiCut : Restrict BLAST to the top X hits from each database [100]

• pagsat: Updated from Version 1.11.2.
→ Version 1.11.3: Added reference=FILE as alias for refgenome=FILE. Fixed orphan delete bug.
→ Version 1.12.0: Tidying up and documenting outputs. Changed default minloclen=250 and minlocid=95. (LTR identification.)
→ Version 2.0.0: Major overhaul of outputs to improve consistency and clarity. Added Snapper to main run.
→ Version 2.1.0: Added localSAM output.
→ Version 2.1.1: Fixed the case of some output files.
→ Version 2.1.2: Fixed some issues with reverse hits in Snapper and application of minlocid.
→ Version 2.2.0: Added mapout=T, which is recommended for first run if going to subsequently tidy. (Run tidy on mapfile.)
→ Version 2.2.1: Tried to fix covplot bug in compare=FILES mode.
→ Version 2.2.2: Cleaned up *.map.* output for SAMPhaser output files. Added tidy/mapfas option selection.
→ Version 2.2.3: Added #NOTE to tidy and fixed makesnp=T bug.
→ Version 2.2.4: Fixed `fragrevcomp=F` bug for Gene and Protein TopHits.
→ Version 2.2.5: Hopefully really fixed makesnp=T bug now!
→ Version 2.2.6: Fixed Haploid tidy sequence output naming bug.
→ Version 2.2.7: Fixed Compare File path bug & dropped some empty outputs.
→ Version 2.3.0: Minor bug fixes and extra tidy options (join gaps and multi-deletes).
→ Version 2.3.1: Minor bug fixes.
→ Version 2.3.2: Updated the synteny mappings to be m::n instead of m:n for Excel compatibility.
→ Version 2.3.3: Fixed bad assembly sequence name bug.

• pagsat_V1: Created/Renamed/moved.
→ Version 1.0.0: Initial working version for based on rje_pacbio assessment=T.
→ Version 1.1.0: Fixed bug with gene and protein summary data. Removed gene/protein reciprocal searches. Added compare mode.
→ Version 1.1.1: Added PAGSAT output directory for tidiness!
→ Version 1.1.2: Renamed the PacBio class PAGSAT.
→ Version 1.2.0: Tidied up output directories. Added QV filter and Top Gene/Protein hits output.
→ Version 1.2.1: Added casefilter=T/F : Whether to filter leading/trailing lower case (low QV) sequences [True]
→ Version 1.3.0: Added tophitbuffer=X and initial synteny analysis for keeping best reference hits.
→ Version 1.4.0: Added chrom-v-contig alignment files along with *.ordered.fas.
→ Version 1.4.1: Made default chromalign=T.
→ Version 1.4.2: Fixed casefilter=F.
→ Version 1.5.0: diploid=T/F : Whether to treat assembly as a diploid [False]
→ Version 1.6.0: mincontiglen=X : Minimum contig length to retain in assembly [1000]
→ Version 1.6.1: Added diploid=T/F to R PNG call.
→ Version 1.7.0: Added tidy=T/F option. (Development)
→ Version 1.7.1: Updated tidy=T/F to include initial assembly.
→ Version 1.7.2: Fixed some bugs introduced by changing gablam fragment output.
→ Version 1.7.3: Added circularise sequence generation.
→ Version 1.8.0: Added orphan processing and non-chr naming of Reference.

Friday, 28 July 2017

Edwards Lab: The SLiMEnrich Shiny App is now live

Edwards Lab: The SLiMEnrich Shiny App is now live: Sobia ’s first Shiny App is now up and running for final pre-publication testing on our new EdwardsLab RShiny server. See post for details.