bioinformatics banner
slider

Getting Started with Pysano

How to get started

To get started with pysano you need to do the following:

  1. Get an account on one of the HCI servers (hci-moab or hci-alta). If you are an HCI employee you already have this.
  2. Acquire some Linux command-line skills. You need to know how to navigate around the Linux file system. This tutorial is a great place to start. And here is another one.
  3. Learn how to edit files on Linux. There are several text editors available on Linux, but the simplest is nano.
  4. Subscribe to the pysano_users email list, to receive news and updates about pysano.
  5. Work through some of the tutorials.

Pysano basics

Job directories

Each pysano job is contained within a Linux directory. Your job directories can be anywhere on the Linux server under your home directory or in /scratch. They may not be located in /tmp or /usr/tmp, since the pysano service does not have access to those locations.

Pysano must have permission to write to your job directories. By default, when you create a new directory on the HCI's Linux server your user group can write to the directory:

$ mkdir new_job_directory
$ ls -ld new_job_directory
drwxrwxr-x 2 bmilash domain^users 4096 Mar  4  2015 new_job_directory

Note the second "rwx" in the permissions for the directory - those are the permissions that pertain to computer users in your user group, which includes pysano. "rwx" indicates that pysano can read from, write to, and execute commands in your job directory. Pysano would not be able to run on this example:

$ ls -ld problem_job_directory
drwxr-xr-x 2 bmilash domain^users 4096 Mar  4  2015 problem_job_directory

Note that the group does not have write permission. You can change these permission with the "chmod" command:

$ chmod g+w problem_job_directory 
$ ls -ld problem_job_directory
drwxrwxr-x 2 bmilash domain^users 4096 Mar  4 13:28 problem_job_directory

Cmd.txt files

Each job directory must contain a script named cmd.txt, and may also contain data files or links to data files that are part of the job. The cmd.txt file is a shell script, a series of shell commands, with a few additions. The cmd.txt file must include your email address in this form:

#e your_email_address@example.edu

Your cmd.txt file may contain regular shell commands, calls to Java .jar files, and pysano macros. Each of these are explained in the reference manual.

Starting and stopping jobs

Once you have created a job directory with a cmd.txt file (and perhaps other files as well) you can start the pysano job with the pstart command:

$ pstart job_dir

Pstart will validate your job directory (checking that the job directory exists and can be written to by pysano, that the cmd.txt file is present and contains your email address, etc) and notify the pysano service about your job. Pysano will then create a script named "pbs.sh" in your job directory with the specific commands that will be executed on the cluster where your job will run. Pstart can start multiple pysano jobs simultaneously:

$pstart job_dir1 job_dir2 job_dir3

If pstart can validate all the job directories then all of them will be started, otherwise none of them will. You will receive an email from pysano for each job that is started. Those emails will include a link to a web page where you can monitor the progress of your jobs.

A running pysano job can be stopped with pstop:

$ pstop job_dir1

You can also run a job as a "dry run" using the pdryrun command. Pysano will validate the job and create the cluster-appropriate script "pbs.sh" in your job directory, but the job itself will not be executed by pysano.

For more details see: