Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-24436

Use obs and camera overrides in make_apdb.py

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Won't Fix
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ap_pipe, ap_verify
    • Labels:
      None

      Description

      make_apdb.py takes an ApPipeConfig and uses it to configure the database. At present, it loads config files requested by the user from the command-line, but does not try to load observatory or camera-specific overrides for the data going into the APDB. This caused ap_verify to run afoul of DM-24435, and may also lead to inconsistent behavior if we ever want to configure the APDB in a per-camera manner (column names for special filters, for example?).

      Modify make_apdb.py to search for and load ap_pipe override files. This requires that make_apdb.py know what camera it is working with. I see two options:

      • provide an input Butler repository, and infer the obs package and camera from the mapper. This has the advantage of common syntax with ap_pipe.py.
      • explicitly name the obs-package and camera on the command line, e.g. "obs_subaru/hsc". This has the advantage of being less confusing for the user, who might otherwise expect that the location of the repository is somehow relevant.

        Attachments

          Issue Links

          blocks

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-24259 Create “stub“ Gen2 HSC dataset for CI testing

          • Done
          relates to

          Bug - A problem which impairs or prevents the functions of the product. DM-24435 Freezing a config locks the registry(ies) of other instances of that config

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-22663 Reimplement make_apdb.py for Gen 3

          • Done

            Activity

            Ascending order - Click to sort in descending order
            krzys Krzysztof Findeisen created issue -
            krzys Krzysztof Findeisen made changes -
            Field Original Value New Value
            Epic Link DM-24341 [ 433028 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue blocks DM-24259 [ DM-24259 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue relates to DM-24435 [ DM-24435 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue relates to DM-22663 [ DM-22663 ]
            krzys Krzysztof Findeisen made changes -
            Issue Type Bug [ 1 ] Improvement [ 4 ]
            krzys Krzysztof Findeisen made changes -
            Description Currently, {{make_apdb.py}} takes an {{ApPipeConfig}} and uses it to configure the database. At present, it loads config files requested by the user from the command-line, but does not try to load observatory or camera-specific overrides for the data going into the APDB. This caused {{ap_verify}} to run afoul of DM-24435, and may also lead to inconsistent behavior if we ever want to configure the APDB in a per-camera manner (column names for special filters, for example?).

            Modify {{make_apdb.py}} to search for and load {{ap_pipe}} override files. This requires that {{make_apdb.py}} know what camera it is working with. I see two options:
            * provide an input Butler repository, and infer the obs package and camera from the mapper. This has the advantage of common syntax with {{ap_pipe.py}}.
            * explicitly name the obs-package and camera on the command line, e.g. {{"obs_subaru/hsc"}}. This has the advantage of being less confusing for the user, who might otherwise expect that the _location_ of the repository is somehow relevant.             		{{make_apdb.py}} takes an {{ApPipeConfig}} and uses it to configure the database. At present, it loads config files requested by the user from the command-line, but does not try to load observatory or camera-specific overrides for the data going into the APDB. This caused {{ap_verify}} to run afoul of DM-24435, and may also lead to inconsistent behavior if we ever want to configure the APDB in a per-camera manner (column names for special filters, for example?).

            Modify {{make_apdb.py}} to search for and load {{ap_pipe}} override files. This requires that {{make_apdb.py}} know what camera it is working with. I see two options:
            * provide an input Butler repository, and infer the obs package and camera from the mapper. This has the advantage of common syntax with {{ap_pipe.py}}.
            * explicitly name the obs-package and camera on the command line, e.g. {{"obs_subaru/hsc"}}. This has the advantage of being less confusing for the user, who might otherwise expect that the _location_ of the repository is somehow relevant.
            krzys Krzysztof Findeisen made changes -
            Description {{make_apdb.py}} takes an {{ApPipeConfig}} and uses it to configure the database. At present, it loads config files requested by the user from the command-line, but does not try to load observatory or camera-specific overrides for the data going into the APDB. This caused {{ap_verify}} to run afoul of DM-24435, and may also lead to inconsistent behavior if we ever want to configure the APDB in a per-camera manner (column names for special filters, for example?).

            Modify {{make_apdb.py}} to search for and load {{ap_pipe}} override files. This requires that {{make_apdb.py}} know what camera it is working with. I see two options:
            * provide an input Butler repository, and infer the obs package and camera from the mapper. This has the advantage of common syntax with {{ap_pipe.py}}.
            * explicitly name the obs-package and camera on the command line, e.g. {{"obs_subaru/hsc"}}. This has the advantage of being less confusing for the user, who might otherwise expect that the _location_ of the repository is somehow relevant.             		{{make_apdb.py}} takes an {{ApPipeConfig}} and uses it to configure the database. At present, it loads config files requested by the user from the command-line, but does not try to load observatory or camera-specific overrides for the data going into the APDB. This caused {{ap_verify}} to run afoul of DM-24435, and may also lead to inconsistent behavior if we ever want to configure the APDB in a per-camera manner (column names for special filters, for example?).

            Modify {{make_apdb.py}} to search for and load {{ap_pipe}} override files. This requires that {{make_apdb.py}} know what camera it is working with. I see two options:
            * -provide an input Butler repository, and infer the obs package and camera from the mapper. This has the advantage of common syntax with {{ap_pipe.py}}.-
            * explicitly name the obs-package and camera on the command line, e.g. {{"obs_subaru/hsc"}}. This has the advantage of being less confusing for the user, who might otherwise expect that the _location_ of the repository is somehow relevant.
            krzys Krzysztof Findeisen made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            krzys Krzysztof Findeisen made changes -
            Component/s ap_pipe [ 14281 ]
            krzys Krzysztof Findeisen made changes -
            Status In Progress [ 3 ] To Do [ 10001 ]
            krzys Krzysztof Findeisen made changes -
            Resolution Done [ 10000 ]
            Status To Do [ 10001 ] Won't Fix [ 10405 ]

              People

              Assignee:
              krzys Krzysztof Findeisen
              Reporter:
              krzys Krzysztof Findeisen
              Watchers:
              Eric Bellm, Krzysztof Findeisen, Meredith Rawls
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.