Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-647

support samples and other many-to-one outputs in SourceRecord

    Details

    • Type: Story
    • Status: Won't Fix
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: afw

      Description

      For DM-645, I need to be able to save multiple samples (each a parameter vector and possibly a weight) with each SourceRecord, with the number of samples possibly differing from SourceRecord to SourceRecord. I'll also want to save Mixture distributions, which can also be easily represented as a sequence-of-tuples of values to be associated with a record.

      Since we don't want to hard-code the outputs of a particular plugin into SourceRecord (which is what I've done with the ModelFitRecord class in meas_multifit, which I'd like to remove), I think this means we want a more general way of allowing algorithms to define many-to-one tables to be associated with each source. While few algorithms will likely want to save samples, this plugin may not be the only one, and it's likely a general many-to-one feature could be used by other algorithms to save diagnostics when configured to do so.

      I'm worried this is another case where afw::table may be intruding on a feature best left to a true RDBMS, but I don't like the idea of tying an RDBMS directly to the source measurement framework either. I'm open to other ideas, if anyone has one.

      It's also worth pointing out that because this is blocking DM-645, which is important for my 20%-time science work, I'd like to get moving on this even before it becomes a real priority for LSST. But I'd still like to get some design feedback on this.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jbosch Jim Bosch
                Reporter:
                jbosch Jim Bosch
                Watchers:
                Jim Bosch, John Swinbank
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel