Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-9172

Implement validation metrics for NMF outputs

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Story Points:
      6
    • Sprint:
      DRP S17-3, DRP S17-4
    • Team:
      Data Release Production

      Description

      Produce test cases for deblender operation on simulated data, including:

      • Correctness of SED;
      • Overall flux;
      • Degenerate templates.

      This should include comparison against the current LSST deblender as a baseline.

        Attachments

          Activity

          Hide
          pmelchior Peter Melchior added a comment -

          OK, proceed with the first item.

          Show
          pmelchior Peter Melchior added a comment - OK, proceed with the first item.
          Hide
          fred3m Fred Moolekamp added a comment -

          I created DM-9784 to implement the overlap test suggested above.

          Show
          fred3m Fred Moolekamp added a comment - I created DM-9784 to implement the overlap test suggested above.
          Hide
          fred3m Fred Moolekamp added a comment -

          Peter Melchior Based on our conversation yesterday I think it is ok to close this ticket. In the meantime we can consider new tests that should be added before we complete DM-9784.

          Show
          fred3m Fred Moolekamp added a comment - Peter Melchior Based on our conversation yesterday I think it is ok to close this ticket. In the meantime we can consider new tests that should be added before we complete DM-9784 .
          Hide
          pmelchior Peter Melchior added a comment -

          Yes, I agree. The problem with the errors is that the model is non-lienar and utilizes constraints on both spatial and spectral parts, so getting proper errors is pretty tough.

          I'll suggest (for later investigation) two approaches:

          • MCMC the variations, starting at the run of the proximal algorithm
          • With the results of DM-9784, we know how accurate the spatial part is. We can then freeze that part, then the NMF becomes a linear problem || Y-AS ||^2, which has an analytic solution and covariance matrix, Sigma_A The constraint that sum_b a_bk = 1 for all objects k, will lead to anti-correlations between the SED amplitude in different bands. I'm pretty sure that one can theoretically predict the correlation matrix C_A_k (based on the SED itself and the noise level in the bands), SVD it, and estimate a SED covariance matrix of object k by rotating Sigma_A_k into the correlated frame.
          Show
          pmelchior Peter Melchior added a comment - Yes, I agree. The problem with the errors is that the model is non-lienar and utilizes constraints on both spatial and spectral parts, so getting proper errors is pretty tough. I'll suggest (for later investigation) two approaches: MCMC the variations, starting at the run of the proximal algorithm With the results of DM-9784 , we know how accurate the spatial part is. We can then freeze that part, then the NMF becomes a linear problem || Y-AS ||^2, which has an analytic solution and covariance matrix, Sigma_A The constraint that sum_b a_bk = 1 for all objects k, will lead to anti-correlations between the SED amplitude in different bands. I'm pretty sure that one can theoretically predict the correlation matrix C_A_k (based on the SED itself and the noise level in the bands), SVD it, and estimate a SED covariance matrix of object k by rotating Sigma_A_k into the correlated frame.
          Hide
          fred3m Fred Moolekamp added a comment - - edited

          merged

          Show
          fred3m Fred Moolekamp added a comment - - edited merged

            People

            • Assignee:
              fred3m Fred Moolekamp
              Reporter:
              swinbank John Swinbank
              Reviewers:
              Peter Melchior
              Watchers:
              Fred Moolekamp, John Swinbank, Peter Melchior
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel