Development News Brief

Get Galaxy

getgalaxy

getgalaxy.org

new: % hg clone http://www.bx.psu.edu/hg/galaxy galaxy-dist

upgrade: % hg pull -u -r 17d57db9a7c0


# EMBOSS Tool Migration

The Emboss 5.0.0 tools, datatypes and functional test data files have been removed from the Galaxy code base in this revision and have been migrated to the Galaxy tool shed. This will affect you only if you are using the Emboss 5.0.0 tools in your local Galaxy environment by having one or more of them defined in you local tool_conf.xml file.


**After you update your Galaxy code base to this revision (if you use any of the [EMBOSS](http://emboss.sourceforge.net) tools), attempting to start your Galaxy server will display a message with instructions on what to do to install the tools from the tool shed.** You can choose to not install the tools and start your Galaxy server - it will start on this second attempt.
tool shed

If you decide to install the tools, the location in which the Emboss repositories will be installed is the value of the "tool_path" attribute in in the <tool> tag of the file named "migrated_tools_conf.xml" in your Galaxy installation directory. The default location setting is "../shed_tools", which may be problematic for some cluster environments, so make sure to change it before you execute the installation process if necessary.

If your local Galaxy environment is configured to run multiple web front-ends, you should start up a single front-end for your Galaxy server and watch its paster log to see the instructions. If you start all configured web front-ends, only the first one started will produce the instructional message, and others will start up your Galaxy server.

At any point, you can install the EMBOSS tools by executing the following from your Galaxy installation directory.

sh ./scripts/migrate_tools/0002_tools.sh

After installation, the tools should be displayed in your Galaxy tool panel in precisely the same way that they were displayed before the installation. In order for this to happen, however, you must have the tools defined in your local tool_conf.xml file as you want them displayed in your tool panel before you perform the installation. After the installation, you can remove the entries for them from your tool_conf.xml file.

We highly recommend that you read the following sections of the Galaxy tool shed wiki (if you have not already done so) to get all of the details about how this process works.

Migrating tools from the Galaxy distribution to the Galaxy Main tool shed


# New Tool Integration Testing

Use Galaxy's functional test framework to test tools installed into your local Galaxy instance from Tool Shed repositories that contain tools with defined functional tests. For tests to work, the repository must include a directory named test-data somewhere in its file system hierarchy, and all functional test input and output datasets must be included in this directory. Read more...


# Updates to Tool Panel Configuration

Galaxy now supports multiple tool panel XML configuration files for populating the tool panel. You can have any number of file names defined in the comma-separated list of tool panel configuration files for the "tool_config_files" setting in your universe_wsgi.ini file.

For example:

tool_config_files = tool_conf.xml,tool_conf1.xml,shed_tool_conf.xml,shed_tool_conf1.xml, etc.

Managing the layout of your Galaxy tool panel is easier! New tool panel items manually added to a tool panel configuration file will now be inserted into an existing integrated_tool_panel.xml file in the proper location. Read more...


# Multiprocess Job Handling

Galaxy has previously been able to split into multiple processes to better spread the load of handling web traffic, but only one process could be used to handle job management. Galaxy can now be split in to any number of web processes, and any number of job handlers. A single job manager must be run to assign jobs to handlers. If you've configured Galaxy with multiple processes as per the "Production Server" documentation, configuration changes will be necessary to use this new functionality. If you do not change your configuration, each web processes will manage its own jobs, which you almost certainly do not want. Read more...


# UCSC Display Sites

If use_remote_user = True in your config, Galaxy previously used a hardcoded list to determine what servers were allowed to bypass security to read datasets for display. This list is now configurable via the display_servers. If you have modified galaxy-dist/lib/galaxy/web/framework/middleware/remoteuser.py to modify the list of hardcoded servers you will most likely experience merge conflicts upon pulling this distribution. If this is the case, you should remove your local modifications and use the display_servers configuration option, which is documented in universe_wsgi.ini.sample.


# Enhanced OpenID Support

Abstract OpenID providers to be defined outside of the Python code and to allow customization of post-authentication actions (currently restricted to storing sreg attributes into user preferences; but more functions can be added as needed). See individual examples in openid/ and the list of enabled OpenID providers in openid_conf.xml.sample to add your own.


# New Configurable Tool Output Location

Tools using the process described at 'Admin/Tools/MultipleOutputFiles' to write multiple output files to the temporary directory defined as new_file_path may now write to the job working directory instead. This can have a performance benefit if there are a lot of files in the directory set in new_file_path. A configuration option, collect_output_from has been added that controls where Galaxy will look for these outputs. By default, it will look in both places since many of the tools in the Galaxy distribution use new_file_path.


# Tools

Admin/Config/Tool Dependencies

  • NGS: GATK Tools (beta)

    • Added GATK version 1.4

      • Compiled from v1.4-18-g80a4ce0
      • Genome Analysis Toolkit (beta) tool group includes: Depth of Coverage on BAM files; Print Reads from BAM files; Realigner Target; Creator for use in local realignment; Indel Realigner - perform local realignment; Count Covariates on BAM files; Table Recalibration on BAM files; Analyze Covariates - draw plots; Unified Genotyper SNP and indel caller; Variant Annotator; Variant Filtration on VCF files; Select Variants from VCF files; Variant Recalibrator; Apply Variant Recalibration; Validate Variants; Eval Variants; Combine Variants.
    • Additional tool documentation developing over next few months. For now, refer to GATK tool author help.
gatk-workflow
* **NGS: Variant Detection** * Added **[FreeBayes](http://github.com/ekg/freebayes)** latest version (a46483351fd0196637614121868fb5c386612b55) * Requires **[SAM Tools](http://samtools.sourceforge.net/), version 0.1.16-0.1.18** *strongly recommended* * **NGS: Mapping** * Correction made for **[BWA](http://bio-bwa.sourceforge.net/) -R** option. * Updated **Megablast** now uses NCBI BLAST+ (changeset [0b5cb60e4810](https://bitbucket.org/galaxy/galaxy-central/changeset/0b5cb60e4810#chg-tools/metag_tools/megablast_wrapper.xml)) * **[BLASTN](http://blast.ncbi.nlm.nih.gov/Blast.cgi) version 2.2.25** * Simplified indexing. Obtain from [NCBI](ftp://ftp.ncbi.nlm.nih.gov/blast/db/) or use [formatdb](ftp://ftp.ncbi.nih.gov/blast/documents/formatdb.html) to create your own. [Read more...](/admin/NGS Local Setup/) * Updated **Parse blast XML output** * Improvements to merging blast xml - `peterjc/galaxy-central/split_blast2` (pull request #41) * **NGS: [SAM Tools](http://samtools.sourceforge.net/)** * Have implicit `SAM to BAM` converter sort the output `BAM` file so that indexing will not fail. * **NGS: RNA-seq** * Updated **[Trinity](http://sourceforge.net/projects/trinityrnaseq/files/)** wrapper (alpha). * Trinity performs de novo assembly of RNA-Seq data. [Read more...](http://trinityrnaseq.sourceforge.net) * Wrapper supports **Trinity r2012-04-22-beta** (and perhaps a few versions earlier). * All commands included, **Inchworm, Chrysalis, and Butterfly**, see `/tools/ngs_rna/trinity_all.xml`. * Updated **[Cuffdiff](http://cufflinks.cbcb.umd.edu/manual.html)** * Galaxy tool form default for `min-alignment-count` is now the same as line command default. * **Get Data** * Update to use **[WormBase 2](http://www.wormbase.org)**. * Correct issues with **[HbVar](http://globin.bx.psu.edu/hbvar/menu.html)**. * **Display Applications** * Added **[Integrated Genome Browser (IGB)](http://bioviz.org/igb/index.html)** as a new display option.
# Tool Shed

Tool Shed


# Galaxy Track Browser (GTB)

Trackster


# Workflows
# User Interface (UI)
# Source

galaxy-cental, galaxy-dist


# Bug Fixes
bugs
* `fasta_to_len` conversion now uses only the first word in the description line from input [FASTA](/learn/datatypes/#fasta) datasets to create the *identifier name* for sequences. This is standards compliant and used by default for automatic conversion. * Add necessary space after threads option in [Cuffmerge](http://cufflinks.cbcb.umd.edu/). * The dataset cleanup scripts will no longer error out if database_connection is not explicitly set in the Galaxy config file. Addresses mailing list questions: * [galaxy-user] Cleanup script error - guess_dialect_for_url * [galaxy-dev] Problem with cleaning up galaxy datasets * Fix for restoring checked state for current value in `DrillDownSelectToolParameter`. * In tool_form (tool_runner), have call to unicode on non unicode field_html use 'replace' instead of raising a server error. Fixes issues seen with non unicode characters in history item names. * Fix two more places where filenames in content-dispositions were not being surrounded by quotes. (Fixes error "349" seen in Chrome browser). * Fix `owner_tag` mapping of `StoredWorkflow`. Previously this mapping incorrectly retrieved *all* workflow tags owned by the user. * Fix inflector for `n == 0`. ("0 line" error). * Fix multi-input workflow selector to account for recent structural changes to the page. * Prevent the history_contents API from adding the HDA to the history twice (didn't duplicate, but it resulted in skipped `hids`). * Update cloud launch config name. * Convert `/galaxy-dist/tool-data/*.loc.sample` files to use tabs instead of spaces in example entries. * Cloud Launch - Password Field for AWS Secret instead of plain text input * Tabular dataset display - Remove logs, revert scroll detection to base. * Screencasts modal size fix, increased default min-width. * Workflow editor fix for the case where `RenameActions` had null action arguments vs being `unspecified` or {} as expected. This situation exists in very few rename actions and hasn't been reproduced, may be legacy workflow steps. * Fix class assignment for links/menus/popups in grids.
# Announcements

News, May 2012 Galaxy Update

GCC2012 Update

2012 Galaxy Community Conference

The 2012 Galaxy Community Conference (GCC2012) will be held July 25-27, in Chicago. The conference will feature two full days of presentations and discussions, and a whole day of training has been added this year.

Early registration is now open and will close June 11. Early registration is very affordable, especially for post-docs and students.

Training Day: Help Set the Agenda

Training Day

The GCC2012 Training Day needs your help! Training Day offers 90 minute training sessions in three parallel tracks throughout the day for a total of 12 sessions. We need your input on how to best allocate those sessions to topics. Please take a few minutes to indicate your top choices for what you would like to see covered on the GCC2012 Training Day.




----

# About Galaxy

GalaxyProject.org

The GalaxyTeam is a part of BX at Penn State, and the Biology and Mathematics and Computer Science departments at Emory University.

Galaxy is supported in part by NSF, NHGRI, the Huck Institutes of the Life Sciences, and The Institute for CyberScience at Penn State, and Emory University.

Join us at Twitter @galaxyproject or just read our tweets Galaxy on Twitter