Working with analyses¶

Once we have a project and at least one event we can start to add analyses. A single analysis with asimov is called a “production”.

Asimov interacts with a number of different analysis pipelines in order to produce configuration files, generate DAGs (files used to launch the jobs on compute clusters), submit those jobs to clusters, and then handle any post-processing of the results.

In order to make starting an analysis as easy as possible asimov maps its standard configuration values to all of its supported pipelines. This means that setting-up equivalent jobs with two different pipelines requires only a couple of commands.

Adding an analysis from a YAML file¶

Todo

Add instructions on using YAML files and the asimov apply command.

Adding a new production¶

Let’s try adding a lalinference production to an event called GW150914. Provided we don’t need to perform any customisation to our production we can just run

$ olivaw production create GW150914 lalinference --approximant IMRPhenomPv2

If we now look at the ledger we can see that the GW150914 event has been updated to include a productions section, and the lalinference job has been added:

events:
- calibration: {}
  event time: 1126259462.426435
  interferometers:
  - H1
  - L1
  name: GW150914
  productions:
  - Prod0:
      approximant: IMRPhenomPv2
      comment: null
      pipeline: lalinference
      review: []
      status: wait
  working directory: working/GW150914

When creating the production we can also add a comment, which may be useful to help distinguish it from similar analyses when we’re looking at many jobs. We can do that by adding the --comment option:

$ olivaw production create GW150914 lalinference --comment "Higher mode analysis"

Naming productions¶

Asimov names productions automatically, with the first becoming Prod0 by default, and future jobs being incremented, so Prod1, Prod2, and so on. We can customise the prefix of the name with the --family option. For example, if you wanted to call your exploratory runs Exp0, Exp1, and so-forth you could do that by using this command:

$ olivaw production create GW150914 lalinference --family Exp

Analysis dependencies¶

When we create the production we can supply some additional information, including a list of dependencies. Any production can have multiple other productions as a dependency; asimov will construct a graph by analysing the dependencies for all productions, and will ensure that they are executed in the correct order.

Suppose you have set up a job which produces PSDs which are required for all subsequent analyses. This production is called Prod0. Then we can set up a subsequent analysis with

$ olivaw production create GW150914 lalinference --needs Prod0

This production will not be set-up and run until Prod0 has been completed.

Note

If you need to add multiple dependencies, just add the --needs option several times, e.g. --needs Prod0 --needs Prod1.

Analysis status¶

Asimov uses its ledger to record the last known state of a production, in the form of a state machine. Details of the possible states are documented on the ../state.rst of the documentation.

By default a new production is assigned a wait state, which prevents asimov from generating the configuration file for the pipeline, or starting the analysis. This is a useful state to put jobs into until you’re happy that all of the settings are correct.

If you want a production to be ready to start as soon as it is created, however, you can pass the --status option, for example

$ olivaw production create GW150914 lalinference --status ready

Pipeline template¶

In order to start an analysis using one of the pipelines asimov must produce an appropriate configuration file for the pipeline generator. These are generated using template files, which are specific to each pipeline. When asimov builds pipelines it substitutes configuration values for each production into these templates.

A default template is included with asimov for each supported pipeline, however there may be situations where these are insufficient and another template is required.

A custom template can be included using the --template option, for example

$ olivaw production create GW150914 lalinference --template testinggr.ini

You can find more information about configuration templates at the templates page of the documentation.