Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-35283

Process Diffim sprint dataset with new Image Differencing

    XMLWordPrintable

Details

    • Story
    • Status: Done
    • Resolution: Done
    • None
    • ip_diffim
    • None
    • 6
    • AP F22-2 (July), AP F22-5 (October), AP F22-6 (November)
    • Alert Production
    • No

    Description

      Use the HSC Cosmos small dataset selected for the 2021 Difference Imaging sprint and process all of the visits in both bands with the new image differencing, in "auto" convolution mode. Include Fake source injection in the pipeline and analysis.

      Attachments

        Issue Links

          Activity

            We've made an initial pass at this step, and while things mostly worked, we saw a 20% failure rate in the calibrate step in r-band due to using GAIA-DR2 as the reference catalog. It's being discussed extensively on slack as well as in DM-27858.

            kherner Kenneth Herner added a comment - We've made an initial pass at this step, and while things mostly worked, we saw a 20% failure rate in the calibrate step in r-band due to using GAIA-DR2 as the reference catalog. It's being discussed extensively on slack as well as in  DM-27858 .

            I think you volunteered for this, sullivan.

            kherner Kenneth Herner added a comment - I think you volunteered for this, sullivan .

            It looks like the "Bright DIA Sources by Collection" plot in the version of the notebook pushed to Github has a bug, and is missing data. The version of that plot you showed at standup included bright DIA sources from each of the collections, and that there was good agreement between the different "flavors" of diffim.

            sullivan Ian Sullivan added a comment - It looks like the "Bright DIA Sources by Collection" plot in the version of the notebook pushed to Github has a bug, and is missing data. The version of that plot you showed at standup included bright DIA sources from each of the collections, and that there was good agreement between the different "flavors" of diffim.

            If you're referring to the second bright source plot, that's actually set to only look for collections with "auto" in the name, and in this particular instance there aren't any, since it's just showing the baseline default settings at the moment, best third seeing templates and convolving the template. The one I showed at standup before Thanksgiving was actually a different notebook (same code, just different list of collections) which I currently have as part of DM-36715. I could certainly change things to move them into this notebook if you think the most recent runs fit better in this ticket.

            kherner Kenneth Herner added a comment - If you're referring to the second bright source plot, that's actually set to only look for collections with "auto" in the name, and in this particular instance there aren't any, since it's just showing the baseline default settings at the moment, best third seeing templates and convolving the template. The one I showed at standup before Thanksgiving was actually a different notebook (same code, just different list of collections) which I currently have as part of DM-36715 . I could certainly change things to move them into this notebook if you think the most recent runs fit better in this ticket.

            sullivan I have updated the notebook with the w46 runs shown at the December QA meeting; feel free to take another look.

            kherner Kenneth Herner added a comment - sullivan I have updated the notebook with the w46 runs shown at the December QA meeting; feel free to take another look.

            People

              kherner Kenneth Herner
              sullivan Ian Sullivan
              Ian Sullivan
              Ian Sullivan, Kenneth Herner
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.