Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11568

SourceDetectionTask's outputs should be contiguous in memory

    XMLWordPrintable

    Details

    • Story Points:
      2
    • Team:
      Data Release Production

      Description

      SourceDetectionTask should be able to put (at least) an upper bound on the number of post-deblending sources it will put into the SourceCatalog it creates, from the size of the FootprintSet it creates and the total number of peaks within it. It should use SourceCatalog.reserve to ensure the resulting catalog is contiguous.

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            As part of the work in DM-9832, I added some "force-contiguous" tests in calibrate.py and multiBand.py, with TODOs referencing this ticket. Once this is implemented, those can probably go away.

            Show
            Parejkoj John Parejko added a comment - As part of the work in DM-9832 , I added some "force-contiguous" tests in calibrate.py and multiBand.py, with TODOs referencing this ticket. Once this is implemented, those can probably go away.
            Hide
            tjenness Tim Jenness added a comment -

            Is this ticket still valid?

            Show
            tjenness Tim Jenness added a comment - Is this ticket still valid?
            Hide
            jbosch Jim Bosch added a comment -

            Yup, still valid, and not very hard to do at all.  I'll add the PairCoding label to try to get it noticed by somebody looking to find a small ticket to work on at Science Pipelines' weekly pair-coding sesssions.

            Show
            jbosch Jim Bosch added a comment - Yup, still valid, and not very hard to do at all.  I'll add the PairCoding label to try to get it noticed by somebody looking to find a small ticket to work on at Science Pipelines' weekly pair-coding sesssions.
            Hide
            Parejkoj John Parejko added a comment -

            Bumped into this while working on DM-38546: I'd love to have this fixed soon, so that we don't have to check contiguity in the new CalibrateImageTask.

            Show
            Parejkoj John Parejko added a comment - Bumped into this while working on DM-38546 : I'd love to have this fixed soon, so that we don't have to check contiguity in the new CalibrateImageTask.
            Hide
            Parejkoj John Parejko added a comment -

            Update to my comment above: the problem I just ran into was post-deblending. It looks like SourceDetectionTask already pre-allocates correctly using sources.reserve on the output of detectFootprints (see this commit from me in 2018!). Whether the deblender should return a contiguous catalog is a different question, but as-written, I think this is actually done.

            Show
            Parejkoj John Parejko added a comment - Update to my comment above: the problem I just ran into was post-deblending. It looks like SourceDetectionTask already pre-allocates correctly using sources.reserve on the output of detectFootprints (see this commit from me in 2018 !). Whether the deblender should return a contiguous catalog is a different question, but as-written, I think this is actually done.
            Hide
            erykoff Eli Rykoff added a comment -

            I'm not sure ... I've seen reserve not do what I thought it should do, and resize is more reliable. Should we also do a final check? In any event, the deblender should definitely return a contiguous catalog.

            Show
            erykoff Eli Rykoff added a comment - I'm not sure ... I've seen reserve not do what I thought it should do, and resize is more reliable. Should we also do a final check? In any event, the deblender should definitely return a contiguous catalog.

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              jbosch Jim Bosch
              Watchers:
              Eli Rykoff, Jim Bosch, John Parejko, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.