Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-22081

Create Decam HiTS 2015 DCR model for ap_pipe

    Details

    • Story Points:
      12
    • Sprint:
      AP F19-6 (November), AP S20-1 (December), AP S20-2 (January)
    • Team:
      Alert Production

      Description

      The 2014 Decam HiTS data we are currently using for ap_pipe and to evaluate the performance of the DCR model has worse seeing and not as even distribution of airmasses as the 2015 data. As a result, using the 2015 data as a template to run ap_pipe on the 2014 may work much better than using the 2014 data as a template for 2015, as we are currently doing. This ticket is to process the 2015 data through to generate DCR models, and run ap_pipe on the 2014 data twice, once using standard coadd templates and once using the DCR templates.

        Attachments

          Issue Links

            Activity

            Hide
            swinbank John Swinbank added a comment -

            Bumping story points on this due to the necessity for extended sanity checking (not yet resolved).

            Show
            swinbank John Swinbank added a comment - Bumping story points on this due to the necessity for extended sanity checking (not yet resolved).
            Hide
            sullivan Ian Sullivan added a comment -

            The 2015 HiTS templates can be found on lsst-dev under /project/sullivan/jan_2020/

            The conclusion from the lengthy notebook is that the 2015 HiTS data does no better for constraining the DCR model, because the high airmass observations that should have helped also have extremely high seeing. If they are included the range of seeing is too much for the current algorithm to handle, and if they are cut then the remaining observations are insufficient to constrain the model.

            I plan to make one more attempt on a new ticket, being far more selective in exactly which observations are included in the model.

            Show
            sullivan Ian Sullivan added a comment - The 2015 HiTS templates can be found on lsst-dev under /project/sullivan/jan_2020/ The conclusion from the lengthy notebook is that the 2015 HiTS data does no better for constraining the DCR model, because the high airmass observations that should have helped also have extremely high seeing. If they are included the range of seeing is too much for the current algorithm to handle, and if they are cut then the remaining observations are insufficient to constrain the model. I plan to make one more attempt on a new ticket, being far more selective in exactly which observations are included in the model.
            Hide
            mrawls Meredith Rawls added a comment -

            This is great! I left some comments on GitHub, but overall I'm happy to have some DCR metric utilities in this funny little repo as well as a very comprehensive notebook showing your results.

            Please add a summary of the main takeaways to the ticket so our future selves don't necessarily have to click through to the notebook. Copy-pasting from the notebook is fine. It'd be handy to attach at least a pair of sources-on-the-sky plots for at-a-glance comparison to this ticket (e.g., DCR or CW with the seeing criterion for 2014-as-template vs. 2015-as-template), too.

            DCR aside, it's a little disconcerting that 2015-as-template has significantly more background/noise/garbage sources than 2014-as-template despite 2015 having overall better seeing. I am moderately amused that you say in the notebook that we should consider throwing out airmass > 2 visits, yet you also wish there was a greater distribution of airmasses to improve DCR. There is a handy figure (Fig 8) in Forster et al. 2016 that shows the airmass distribution of the HiTS campaigns, which I trust you've seen, and there are a few 2014 visits with airmass > 2, but I don't think there are any 2015 ones. Both the 2014 and 2015 visits have pretty bimodal airmass distributions. I guess I would naively assume the airmass > 2 visits are the first to go when we impose a good seeing criterion, but maybe not.

            Show
            mrawls Meredith Rawls added a comment - This is great! I left some comments on GitHub, but overall I'm happy to have some DCR metric utilities in this funny little repo as well as a very comprehensive notebook showing your results. Please add a summary of the main takeaways to the ticket so our future selves don't necessarily have to click through to the notebook. Copy-pasting from the notebook is fine. It'd be handy to attach at least a pair of sources-on-the-sky plots for at-a-glance comparison to this ticket (e.g., DCR or CW with the seeing criterion for 2014-as-template vs. 2015-as-template), too. DCR aside, it's a little disconcerting that 2015-as-template has significantly more background/noise/garbage sources than 2014-as-template despite 2015 having overall better seeing. I am moderately amused that you say in the notebook that we should consider throwing out airmass > 2 visits, yet you also wish there was a greater distribution of airmasses to improve DCR. There is a handy figure (Fig 8) in Forster et al. 2016 that shows the airmass distribution of the HiTS campaigns, which I trust you've seen, and there are a few 2014 visits with airmass > 2, but I don't think there are any 2015 ones. Both the 2014 and 2015 visits have pretty bimodal airmass distributions. I guess I would naively assume the airmass > 2 visits are the first to go when we impose a good seeing criterion, but maybe not.
            Hide
            mrawls Meredith Rawls added a comment -

            (My Jira hadn't refreshed, and I see now you did summarize at least the DCR conclusions from the notebook, so you may ignore that part of my previous comment!)

            Show
            mrawls Meredith Rawls added a comment - (My Jira hadn't refreshed, and I see now you did summarize at least the DCR conclusions from the notebook, so you may ignore that part of my previous comment!)
            Hide
            mrawls Meredith Rawls added a comment -

            A quick note to both Ian Sullivan and John Swinbank, can we please avoid the term "sanity check" when possible, as it is ableist (this is easily googleable if you want to read up on why). An alternative might be "quick check" or "initial check" or "rationality check," or just a short description of what is actually being checked. Thanks!

            Show
            mrawls Meredith Rawls added a comment - A quick note to both Ian Sullivan and John Swinbank , can we please avoid the term "sanity check" when possible, as it is ableist (this is easily googleable if you want to read up on why). An alternative might be "quick check" or "initial check" or "rationality check," or just a short description of what is actually being checked. Thanks!

              People

              • Assignee:
                sullivan Ian Sullivan
                Reporter:
                sullivan Ian Sullivan
                Reviewers:
                Meredith Rawls
                Watchers:
                Eric Bellm, Ian Sullivan, John Swinbank, Meredith Rawls
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: