Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-31462

Add random seed config to make fgcm density downsampling repeatable

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: fgcmcal
    • Labels:
      None

      Description

      In gen2/gen3 comparisons on w30, there were some differences between the fgcmcal outputs. I have tracked a large part of this to the fgcm high density region downsampling, where stars to use as reference are randomly selected in regions of high density above a configurable threshold. This ticket is to add the functionality to specify a random seed to ensure that the same stars are selected in subsequent or concurrent runs.

        Attachments

          Issue Links

          mentioned in

          Page Loading...

          Page Loading...

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            erykoff Eli Rykoff added a comment -

            I have also updated the gen2 RC2 configuration scripts on this ticket to match what I believe was run on w_2021_30 rerun by Morgan Schmitz.

            With these changes we get the same star list for gen2 and gen3.

            PRs to review:
            https://github.com/lsst-dm/gen2gen/pull/4 (gen2)
            https://github.com/lsst/obs_subaru/pull/379 (gen3)
            https://github.com/lsst/fgcmcal/pull/66 (fgcmcal)
            PR to approve but doesn't need code checks:
            https://github.com/lsst/fgcm/pull/23

            Jenkins is here: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/34801/pipeline

            Show
            erykoff Eli Rykoff added a comment - I have also updated the gen2 RC2 configuration scripts on this ticket to match what I believe was run on w_2021_30 rerun by Morgan Schmitz . With these changes we get the same star list for gen2 and gen3. PRs to review: https://github.com/lsst-dm/gen2gen/pull/4 (gen2) https://github.com/lsst/obs_subaru/pull/379 (gen3) https://github.com/lsst/fgcmcal/pull/66 (fgcmcal) PR to approve but doesn't need code checks: https://github.com/lsst/fgcm/pull/23 Jenkins is here: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/34801/pipeline
            Hide
            Permalink
            lauren Lauren MacArthur added a comment -

            Thank you so much for tracking this down!  Also huge thanks for updating the gen2gen files.  I left the most minor of comments on the PRs, but otherwise LGTM on green Jenkins.

            Show
            lauren Lauren MacArthur added a comment - Thank you so much for tracking this down!  Also huge thanks for updating the gen2gen files.  I left the most minor of comments on the PRs, but otherwise LGTM on green Jenkins.
            Hide
            Permalink
            erykoff Eli Rykoff added a comment -

            Turns out the PR on fgcm master was not the right thing. I've fixed that up and now Jenkins is GREEN. https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/34804/pipeline

            Show
            erykoff Eli Rykoff added a comment - Turns out the PR on fgcm master was not the right thing. I've fixed that up and now Jenkins is GREEN. https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/34804/pipeline

              People

              Assignee:
              erykoff Eli Rykoff
              Reporter:
              erykoff Eli Rykoff
              Reviewers:
              Lauren MacArthur
              Watchers:
              Eli Rykoff, Lauren MacArthur
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.