Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-35283

Process Diffim sprint dataset with new Image Differencing

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ip_diffim
    • Labels:
      None
    • Story Points:
      6
    • Sprint:
      AP F22-2 (July), AP F22-5 (October), AP F22-6 (November)
    • Team:
      Alert Production
    • Urgent?:
      No

      Description

      Use the HSC Cosmos small dataset selected for the 2021 Difference Imaging sprint and process all of the visits in both bands with the new image differencing, in "auto" convolution mode. Include Fake source injection in the pipeline and analysis.

        Attachments

          Issue Links

            Activity

            Hide
            kherner Kenneth Herner added a comment -

            We've made an initial pass at this step, and while things mostly worked, we saw a 20% failure rate in the calibrate step in r-band due to using GAIA-DR2 as the reference catalog. It's being discussed extensively on slack as well as in DM-27858.

            Show
            kherner Kenneth Herner added a comment - We've made an initial pass at this step, and while things mostly worked, we saw a 20% failure rate in the calibrate step in r-band due to using GAIA-DR2 as the reference catalog. It's being discussed extensively on slack as well as in  DM-27858 .
            Hide
            kherner Kenneth Herner added a comment -

            I think you volunteered for this, Ian Sullivan.

            Show
            kherner Kenneth Herner added a comment - I think you volunteered for this, Ian Sullivan .
            Hide
            sullivan Ian Sullivan added a comment -

            It looks like the "Bright DIA Sources by Collection" plot in the version of the notebook pushed to Github has a bug, and is missing data. The version of that plot you showed at standup included bright DIA sources from each of the collections, and that there was good agreement between the different "flavors" of diffim.

            Show
            sullivan Ian Sullivan added a comment - It looks like the "Bright DIA Sources by Collection" plot in the version of the notebook pushed to Github has a bug, and is missing data. The version of that plot you showed at standup included bright DIA sources from each of the collections, and that there was good agreement between the different "flavors" of diffim.
            Hide
            kherner Kenneth Herner added a comment -

            If you're referring to the second bright source plot, that's actually set to only look for collections with "auto" in the name, and in this particular instance there aren't any, since it's just showing the baseline default settings at the moment, best third seeing templates and convolving the template. The one I showed at standup before Thanksgiving was actually a different notebook (same code, just different list of collections) which I currently have as part of DM-36715. I could certainly change things to move them into this notebook if you think the most recent runs fit better in this ticket.

            Show
            kherner Kenneth Herner added a comment - If you're referring to the second bright source plot, that's actually set to only look for collections with "auto" in the name, and in this particular instance there aren't any, since it's just showing the baseline default settings at the moment, best third seeing templates and convolving the template. The one I showed at standup before Thanksgiving was actually a different notebook (same code, just different list of collections) which I currently have as part of DM-36715 . I could certainly change things to move them into this notebook if you think the most recent runs fit better in this ticket.
            Hide
            kherner Kenneth Herner added a comment -

            Ian Sullivan I have updated the notebook with the w46 runs shown at the December QA meeting; feel free to take another look.

            Show
            kherner Kenneth Herner added a comment - Ian Sullivan I have updated the notebook with the w46 runs shown at the December QA meeting; feel free to take another look.

              People

              Assignee:
              kherner Kenneth Herner
              Reporter:
              sullivan Ian Sullivan
              Reviewers:
              Ian Sullivan
              Watchers:
              Ian Sullivan, Kenneth Herner
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.