Development News Brief

Get Galaxy

getgalaxy    getgalaxy.org
new: $ hg clone http://www.bx.psu.edu/hg/galaxy galaxy-dist
upgrade: $ hg pull -u -r 5dcbbdfe1087


Picard Migration to Tool Shed

The tools in the group NGS: Picard (beta) have moved from the Galaxy distribution to the Galaxy Main Tool Shed.

tool shed

Migration scripts will run upon Galaxy's first launch (after updating to this release) that will automatically handle installing the Picard tool wrappers from the Tool Shed. Picard itself and target reference genomes should still be installed as described in the Galaxy wiki - start in the Tool Dependencies section.

Picard comprises Java-based command-line utilities that manipulate SAM files, and a Java API (SAM-JDK) for creating new programs that read and write SAM files. Both SAM text format and SAM binary (BAM) format are supported. More about SAM and BAM format.


## Galaxy Code Documentation Now Available
Sphinx Python Documentation Generator

The Galaxy Project is now using the Sphinx Python document generator to automatically generate documentation for the code base. The documentation describes classes and methods (and much more) in the code base. It also includes Python docstrings from the code.

Two versions of the documentation are available:

Read the Docs

Both versions are hosted at Read the Docs, a community supported web site for code documentation.

If you have made your own local extensions to Galaxy, you can also generate your own local copy of the documentation by going to your doc directory and entering

make html

Please note that this is the first release of this documentation and it is far from where we want it to be. We aim to have the documentation improve incrementally with every new distribution.


# Tool Shed

Tool Shed

Tool Shed Enhancements inside of Galaxy

  1. When migrating tools:

    • Fail gracefully if install_dependencies is requested, but tool_dependency_dir is not set.
    • If only some tools will be installed, list the ones that won't be enabled.

A single tool or a suite of tools per repository ?

Many tool developers in the Galaxy community question the best way to organize tools in their tool shed repositories. Some groups have developed a large number of tools to allow their labs to perform analyses in Galaxy and took the approach of including all related tools in a single repository in the Tool Shed. Others have chosen to restrict each repository to include a single tool.

Which is the *** "best practice" *** ? Read more….

New tool shed repository features

  1. The Intergalactic Utilities Commission will soon begin reviewing repositories in the Galaxy Main Tool Shed
The **Intergalactic Utilities Commission (IUC)** was established in late 2012 to enable the pervasive use of the **[Galaxy Main Tool Shed](http://toolshed.g2.bx.psu.edu/)** by ensuring the repositories available in the tool shed include contents that are functionally correct and optimized for installation into **local Galaxies**. When appropriate, guidance is provided to repository owners so that they can understand what changes are necessary for their repository to be approved.

The features described in this document enable tool shed repositories to ultimately be flagged as "Approved" by the IUC. If a repository is ** "IUC Approved" **, then those installing the repository should have a certain level of comfort that it's contents are functionally correct in all appropriate areas. Read more…

1. #2 Marking a repository you own as deprecated
Certain repository contents may occasionally become outdated, perhaps due to other repositories with tools that replace the outdated tools. The tool shed enables repository owners to mark a repository as deprecated if they so choose. Many of the features available to repositories not marked as deprecated are eliminated from repositories that are, and deprecated repositories are not included in most tool shed repository lists or searches. However, those that installed a repository into their local Galaxy instance before it was marked as deprecated can get appropriate updates to their locally installed repository. [Read more…](http://wiki.galaxyproject.org/ToolShedRepositoryFeatures#Marking_a_repository_you_own_as_deprecated).

Miscellaneous tool shed enhancements and fixes

  1. Enhancements to the tool_config_file; thanks to John Chilton)

    • Galaxy now loads tool sections from multiple files in a more consistent manner, and
    • Directories can now appear as a tool_config_file option
  2. Allow non-admin users to see metadata revisions when viewing a repository changelog in the tool shed.
  3. Add help text to the tool shed repository upload form to advise contributors of recent enhancement allowing for mercurial repository urls staring with hg:// or hgs:// ; inspired by John Chilton.
  4. New support for installation of tool dependencies that are zip archives when installing repositories from the tool shed; contributed by Björn Grüning.
Here's an example `` tag in `tool_dependencies.xml` showing this new supported type: ``` http://downloads.sourceforge.net/project/picard/picard-tools/1.56/picard-tools-1.56.zip ```
5. #5 Fix for determining the extraction directory for tool dependency tarballs when installing tool dependencies along with tool shed repositories. 6. #6 HTML escape values that could be set by the user, prevents XSS.
# Tools

Tool Dependencies

  1. Enable Tophat, Tophat2, and Cuffdiff to accept gene annotations in GFF3 format; thanks to Jim Johnson.
  2. Make unordered GTF parsing more lenient by default, allowing for parsing of 'imperfect' GTF datasets not in strict specification format.
  3. New warning given when loading a .. / tool-data / *.loc file with an inconsistent numbers of tabs; contributed by Peter Cock.
  4. Allow Rerun to access hidden datasets.
  5. Enhance GenomeSpace import tools to display a better history item name.

# API
  1. api/users.show: added 'current' as viable id to display json for the current transaction's user.
  2. api/users.show, index: added key 'quota_percent' to json, returns null if no quota on user, percent otherwise.
  3. api/history_contents: allow anon-user to query own, current history; improve doc string.
  4. api/history_contents.index: if passed a comma separated list of encoded hda ids on the query string (e.g. ids=id1,id2,...) in query string, provides full data (as history_contents.show) for each id listed. If 'ids' isn't used: returns 'summary' behavior as before.
  5. api/history_contents.show: return 'accessible', 'api_type' (consistent with summary style in index), 'display_types', 'display_apps', 'file_ext', 'hid', 'history_id', 'meta_files' (file types of associated files for generating download urls, eg. myBam.meta_files: [{ 'file_type' : 'bam_index' }]), 'visualizations' (list of visualization names appr. for this hda), 'peek' (data.peek as used in the history panel)
  6. api/histories.show: added 'nice_size', 'annotation', 'state_ids' (map of possible hda states : lists of hda ids in those states ), reorder history state from child hda states (running/queued higher priority than error).

# Source
  1. Reloading a tool now automatically populates the field for resubmission, making it easier to continually reload a tool when developing. Pull Request: Tool reload improvements, #72; contributed by John Chilton.
  2. Tabular Display: 'list' column type is now left-aligned, making it easier to view certain datatypes.
  3. Cloudlaunch: Automatically detect placement AZ of volumes from existing clusters, and launch using those placements. This prevents any * "CRITICAL: This instance must be relaunched in zone..." * at Cloudman start. Also fixed a bug where non-keyboard paste wasn't detected during the cluster lookup step.

# Security Fixes

All Galaxy instance maintainers are strongly encouraged to run the latest release.

  1. Recent library browsing optimizations were retracted due to minor security vulnerability: a UI bug that allowed users to see listings of inaccessible datasets. Access was limited to viewing named dataset lists, no "peek" or "full" dataset viewing, download, coping, or any other privileges were granted. Once corrected, the library enhancements will be re-released in a future distribution.
  2. Prevent potential reflected XSS via MessageException (possible in the case of invalid/malicious id lookups, for instance).
  3. Sanitize grid filters.

# Bug Fixes
  1. Tweak for functional tests of BAM outputs when run on a Macintosh OS.

# Announcements

News, November 2012 Galaxy Update

Highlights

Phylotastic Hackathon
* There are now [over 500 papers in the Galaxy CiteULike Group](http://www.citeulike.org/group/16008/library) * Phylogenies! Hacking! Tucson in January! And, hopefully Galaxy too. If you are interested in coding, phylogenies, and tool integration please take a look at the [Phylotastic Hackathon Announcement](https://lists.galaxyproject.org/archives/list/galaxy-dev@lists.galaxyproject.org/thread/PS4SY66TLUILQM6CZC7DXT52RPESVHYJ/#PS4SY66TLUILQM6CZC7DXT52RPESVHYJ). Apply by ***November 16*** (that's this *Friday!!*).

Find a great Galactic job!

Please Help!  Yes you!

Got a Galaxy-related opening? Send it to outreach@galaxyproject.org.



# About Galaxy

GalaxyProject.org

The Galaxy Team is a part of BX at Penn State, and the Biology and Mathematics and Computer Science departments at Emory University.

Galaxy is supported in part by NSF, NHGRI, the Huck Institutes of the Life Sciences, and The Institute for CyberScience at Penn State, and Emory University.

Join us at Twitter @galaxyproject or just read our tweets [Galaxy on Twitter](http://wiki.galaxyproject.org/Galaxy on Twitter)