Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-2991

Firefly External Task Launcher

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: SUIT
    • Labels:
      None

      Description

      Implement and external task launcher, which forks a [python] process and gets back the results. The results can be a table, an image, or a JSON.

        Attachments

          Issue Links

            Activity

            Hide
            tatianag Tatiana Goldina added a comment -

            External task is defined by 3 things:

            • location of the executable (launcher)
            • task name
            • task parameters

            To define an external launcher in Firefly
            Define executable in app.prop, using the following convention:
            launchername.exe = @launchername.exe@

            These are the implementation details that place certain requirements on the python launcher implementation:

            1. Task parameters are passed via json file. The json input is coming directly from user: it's JSON.stringify from "taskParams" object below.

            function onFireflyLoaded() {
            var tableData= { "processor" : "TableFromExternalTask",
            "launcher" : "python",
            "task" : "TestTask",
            "taskParams" :

            { "param1" : "str-1", "param2" : 23456 }

            };
            firefly.showTable(tableData, "tableHere");

            2. All other parameters as passed via key/value pairs or options. This way there is no danger that the order or extra parameters will affect the existing behavior.

            This is the list of currently used options:
            -d DIR, --work=DIR work directory
            -i FILE, --in=FILE json file with task params
            -n TASK, --name=TASK task name (no spaces)
            -o DIR, --outdir=DIR directory for the final output file

            In future (requested by Gregory):
            -s STR, --sep=STR separator string, after which task status is written (default to "__TASK STATUS__")

            3. A suggested output directory is provided rather than a suggested output file. It's up to the python launcher to create a unique file in this directory.

            4. What is expected to be in standard error and standard output stream

            • STDERR (Standard Error Stream) - whatever is coming from it is logged as warnings for now - we might come with a better idea later.
            • STDOUT (Standard Output Stream) - can contain debugging info, the final output must be a line with the keyword, followed by JSON, which contains error message if any.

            __TASK STATUS__

            { outfile: "/path/file.fits" }

            OR

            __TASK STATUS__

            { error: "Description of the error" }

            5. External process exit status 0 means the execution was normal. Everything else means an error was encountered.

            Show
            tatianag Tatiana Goldina added a comment - External task is defined by 3 things: location of the executable (launcher) task name task parameters To define an external launcher in Firefly Define executable in app.prop, using the following convention: launchername.exe = @launchername.exe@ These are the implementation details that place certain requirements on the python launcher implementation: 1. Task parameters are passed via json file. The json input is coming directly from user: it's JSON.stringify from "taskParams" object below. function onFireflyLoaded() { var tableData= { "processor" : "TableFromExternalTask", "launcher" : "python", "task" : "TestTask", "taskParams" : { "param1" : "str-1", "param2" : 23456 } }; firefly.showTable(tableData, "tableHere"); 2. All other parameters as passed via key/value pairs or options. This way there is no danger that the order or extra parameters will affect the existing behavior. This is the list of currently used options: -d DIR, --work=DIR work directory -i FILE, --in=FILE json file with task params -n TASK, --name=TASK task name (no spaces) -o DIR, --outdir=DIR directory for the final output file In future (requested by Gregory): -s STR, --sep=STR separator string, after which task status is written (default to "__ TASK STATUS __") 3. A suggested output directory is provided rather than a suggested output file. It's up to the python launcher to create a unique file in this directory. 4. What is expected to be in standard error and standard output stream STDERR (Standard Error Stream) - whatever is coming from it is logged as warnings for now - we might come with a better idea later. STDOUT (Standard Output Stream) - can contain debugging info, the final output must be a line with the keyword, followed by JSON, which contains error message if any. __ TASK STATUS __ { outfile: "/path/file.fits" } OR __ TASK STATUS __ { error: "Description of the error" } 5. External process exit status 0 means the execution was normal. Everything else means an error was encountered.

              People

              • Assignee:
                tatianag Tatiana Goldina
                Reporter:
                tatianag Tatiana Goldina
                Watchers:
                Tatiana Goldina
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Due:
                  Created:
                  Updated:
                  Resolved: