Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-29820

Compare Gen2 vs. Gen3 fgcm photoCalibs up to w_2022_04 HSC-RC2 run

    XMLWordPrintable

    Details

    • Story Points:
      8
    • Epic Link:
    • Sprint:
      DRP S21a (Dec Jan), DRP S21b, DRP S22A
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      As part of our journey towards deprecation of the "gen2" middleware in favor of "gen3", compare the fgcm external photoCalib objects from our first full RC2 run on both platforms (performed with the w_2021_14 stack).

        Attachments

          Issue Links

            Activity

            Hide
            lauren Lauren MacArthur added a comment -
            Show
            lauren Lauren MacArthur added a comment - Oh, and FYI, the big offset is "new", as in it was much smaller (almost zero, in fact) in the previous run: https://lsst.ncsa.illinois.edu/~lauren/HSC_RC2/w_2021_46/vsGen3/plots/HSC-G/tract-9697/visit-34384/compareVisit-v34384-diff_base_CircularApertureFlux_12_0-psfMagHist.png
            Hide
            erykoff Eli Rykoff added a comment -

            Bad news and good news.

            I did a comparison between the fgcm outputs for the standard stars on gen2/gen3 on w_2022_04 and it looks like complete garbage:

            But I also did the same comparison between the standard stars on gen3 w_2021_50 and gen3 w_2022_04 and they look just fine:

            So whatever has gone terribly wrong seems to have gone wrong on the gen2 side. I do not think it is worth keeping gen2 around to investigate what has gone wrong on that end. (Hopefully Lauren MacArthur agrees.). However, it is vital that we monitor the statistics from run to run in gen3 in case there is some transient issue that just happened to hit in gen2. On the other hand, I think it's more likely that I inadvertently broke something on the gen2 side (not that there's any smoking gun, it's just that debugging gen2 does not seem like the best use of time.)

            Show
            erykoff Eli Rykoff added a comment - Bad news and good news. I did a comparison between the fgcm outputs for the standard stars on gen2/gen3 on w_2022_04 and it looks like complete garbage: But I also did the same comparison between the standard stars on gen3 w_2021_50 and gen3 w_2022_04 and they look just fine: So whatever has gone terribly wrong seems to have gone wrong on the gen2 side. I do not think it is worth keeping gen2 around to investigate what has gone wrong on that end. (Hopefully Lauren MacArthur agrees.). However, it is vital that we monitor the statistics from run to run in gen3 in case there is some transient issue that just happened to hit in gen2. On the other hand, I think it's more likely that I inadvertently broke something on the gen2 side (not that there's any smoking gun, it's just that debugging gen2 does not seem like the best use of time.)
            Hide
            lauren Lauren MacArthur added a comment -

            Lauren MacArthur agrees

            Show
            lauren Lauren MacArthur added a comment - Lauren MacArthur agrees
            Hide
            erykoff Eli Rykoff added a comment -

            I guess this is reviewed then?

            Show
            erykoff Eli Rykoff added a comment - I guess this is reviewed then?
            Hide
            lauren Lauren MacArthur added a comment -

            I agree again!  Thanks, Eli.  I will update the confluence page to note our collective assessment.

            Show
            lauren Lauren MacArthur added a comment - I agree again!  Thanks, Eli.  I will update the confluence page to note our collective assessment.

              People

              Assignee:
              lauren Lauren MacArthur
              Reporter:
              lauren Lauren MacArthur
              Reviewers:
              Eli Rykoff
              Watchers:
              Eli Rykoff, Jim Bosch, Lauren MacArthur, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.