Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16360

Estimate expected density of reference sources from Gaia

    XMLWordPrintable

    Details

    • Team:
      DM Science

      Description

      Understand the distribution of and peak densities of reference catalogs possible in Gaia.

        Attachments

          Issue Links

          blocks

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-16361 Optimize memory usage in MatchPessimisticB

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            ctslater Colin Slater added a comment -

            In DM-15921, we noticed excessive memory usage when the loaded refcat had ~37k sources covering a single Decam CCD (with surrounding buffer area). This density corresponds to roughly 700k/sq deg. Taking the entire Gaia DR2 catalog (with no cuts), this density is met or exceeded in 0.5% of sky area.

            The matcher memory usage is proportional to N^2, so if the refcat density is closer to 10k/CCD (190k/sq deg), the memory usage is reduced by a factor of 13. Approximately 5% of the reference catalog area exceeds this density.

            Show
            ctslater Colin Slater added a comment - In DM-15921 , we noticed excessive memory usage when the loaded refcat had ~37k sources covering a single Decam CCD (with surrounding buffer area). This density corresponds to roughly 700k/sq deg. Taking the entire Gaia DR2 catalog (with no cuts), this density is met or exceeded in 0.5% of sky area. The matcher memory usage is proportional to N^2, so if the refcat density is closer to 10k/CCD (190k/sq deg), the memory usage is reduced by a factor of 13. Approximately 5% of the reference catalog area exceeds this density.
            Hide
            Permalink
            ctslater Colin Slater added a comment -

            I think I've provided all that was asked for.

            Show
            ctslater Colin Slater added a comment - I think I've provided all that was asked for.

              People

              Assignee:
              ctslater Colin Slater
              Reporter:
              cmorrison Chris Morrison [X] (Inactive)
              Reviewers:
              Chris Morrison [X] (Inactive)
              Watchers:
              Chris Morrison [X] (Inactive), Colin Slater, Eric Bellm, John Swinbank
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.