Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11962

Finalize Timeseries features to be computed on DIAObjects

    Details

      Description

      The DPDD specifies that DIAObjects will have periodic and non-periodic timeseries features computed on them. These are baselined to be those in Richards et al. 2011, but the DPDD is clear that the choice of features should be re-examined closer to operations to incorporate new developments in the field.

      This ticket is to update the DPDD with new features developed in consultation with the science collaborations.

        Attachments

          Issue Links

            Activity

            Hide
            ebellm Eric Bellm added a comment -
            Show
            ebellm Eric Bellm added a comment - cf. http://cesium-ml.org/
            Hide
            mgraham Melissa Graham added a comment -

            The Transients and Variable Stars Science Collaboration has a Taskforce with the purpose of "Review variability characterization parameters planned for the Level 1 and Level 2 LSST catalogs". The Taskforce was formed and charged in April 2018. They have had one telecon in April 2018 in which they chose a spokesperson, created a Slack channel (#tvs_taskforce_var_par), and created a webpage to share information and track their progress. They are starting to gather science cases and make a plan with this TVS Taskforce VarPar Gantt Chart, but it remains mostly unpopulated. I attempted to help out by sending the Taskforce an email with a breakdown of DM's existing plans for Variability Characterization, with references to DM's documentation, but didn't ever hear a reply (attached as an ascii file). I'm currently unsure of their timeline but have inquired on their Slack channel.

            Show
            mgraham Melissa Graham added a comment - The Transients and Variable Stars Science Collaboration has a Taskforce with the purpose of "Review variability characterization parameters planned for the Level 1 and Level 2 LSST catalogs". The Taskforce was formed and charged in April 2018. They have had one telecon in April 2018 in which they chose a spokesperson, created a Slack channel (#tvs_taskforce_var_par), and created a webpage to share information and track their progress. They are starting to gather science cases and make a plan with this TVS Taskforce VarPar Gantt Chart , but it remains mostly unpopulated. I attempted to help out by sending the Taskforce an email with a breakdown of DM's existing plans for Variability Characterization, with references to DM's documentation, but didn't ever hear a reply (attached as an ascii file). I'm currently unsure of their timeline but have inquired on their Slack channel.
            Hide
            mgraham Melissa Graham added a comment -

            Note this related ticket: "is float[6x32] adequate for Object variability parameters with special programs?" https://jira.lsstcorp.org/browse/DM-12581

            Show
            mgraham Melissa Graham added a comment - Note this related ticket: "is float [6x32] adequate for Object variability parameters with special programs?" https://jira.lsstcorp.org/browse/DM-12581
            Hide
            mgraham Melissa Graham added a comment -

            This work is likely to yield a future document, e.g., a DMTN, that describes the adopted variability characterization parameters that are calculated by the Prompt and/or DR pipelines (although this information would also be in LDM-151, a short specific document that is appropriate for the science community would be good as well). That document would likely satisfy PST-31.

            Show
            mgraham Melissa Graham added a comment - This work is likely to yield a future document, e.g., a DMTN, that describes the adopted variability characterization parameters that are calculated by the Prompt and/or DR pipelines (although this information would also be in LDM-151, a short specific document that is appropriate for the science community would be good as well). That document would likely satisfy PST-31.
            Hide
            mgraham Melissa Graham added a comment -

            (1) It appears that the TVS Task Force convened to explore Variability Characterization Parameters (VCP) is currently (and temporarily) dormant as all efforts are focused on the Cadence Optimization White Papers. Eric Bellm, I think we should reset the deadline to e.g., late 2019.

            (2) When this study resumes, limits on the amount of Prompt processing that could/would be allotted to calculating VCP will be probably needed, so that the task force can appropriately prioritize the different VCP (otherwise they might end up with an unattainable wish list). I'll ping KT about this to see if any numbers currently exist, and report back.

            Show
            mgraham Melissa Graham added a comment - (1) It appears that the TVS Task Force convened to explore Variability Characterization Parameters (VCP) is currently (and temporarily) dormant as all efforts are focused on the Cadence Optimization White Papers. Eric Bellm, I think we should reset the deadline to e.g., late 2019. (2) When this study resumes, limits on the amount of Prompt processing that could/would be allotted to calculating VCP will be probably needed, so that the task force can appropriately prioritize the different VCP (otherwise they might end up with an unattainable wish list). I'll ping KT about this to see if any numbers currently exist, and report back.
            Hide
            ebellm Eric Bellm added a comment -

            This page https://www.kaggle.com/michaelapers/the-plasticc-astronomy-classification-demo provides a useful list of existing relevant classification work as well as feature construction libraries. It strikes me that the output of PLAsTiCC should be useful in guiding this study.

            Show
            ebellm Eric Bellm added a comment - This page https://www.kaggle.com/michaelapers/the-plasticc-astronomy-classification-demo provides a useful list of existing relevant classification work as well as feature construction libraries. It strikes me that the output of PLAsTiCC should be useful in guiding this study.
            Hide
            ebellm Eric Bellm added a comment -

            The TVS Task Force has apparently become active but is approaching the problem by considering a small handful of object types, which I do not think is the best approach.

            I note that the winning entry to PLAsTiCC used GPR to flexibly parameterize lightcurve behavior (e.g., the existing and measurement of peaks) in a way that may be useful when considering non-periodic features for transients.

            I note that there may be some tension here between providing precomputed features that are valuable as inputs to machine-learned models, and features that may be more interpretable and hence useful for human querying and filtering of DIAObjects.

            Show
            ebellm Eric Bellm added a comment - The TVS Task Force has apparently become active but is approaching the problem by considering a small handful of object types, which I do not think is the best approach. I note that the winning entry to PLAsTiCC used GPR to flexibly parameterize lightcurve behavior (e.g., the existing and measurement of peaks) in a way that may be useful when considering non-periodic features for transients. I note that there may be some tension here between providing precomputed features that are valuable as inputs to machine-learned models, and features that may be more interpretable and hence useful for human querying and filtering of DIAObjects.
            Hide
            ebellm Eric Bellm added a comment -

            (continuing to capture notes here: some transient-specific features in http://adsabs.harvard.edu/abs/2019arXiv190400014M)

            Show
            ebellm Eric Bellm added a comment - (continuing to capture notes here: some transient-specific features in http://adsabs.harvard.edu/abs/2019arXiv190400014M )
            Hide
            ebellm Eric Bellm added a comment -

            Noting here because I didn't realize it earlier: LDM-151 section 6.21 has an enumerated set of baselined periodic and non-periodic features (again, derived from Butler et al. 2011).

            Show
            ebellm Eric Bellm added a comment - Noting here because I didn't realize it earlier: LDM-151 section 6.21 has an enumerated set of baselined periodic and non-periodic features (again, derived from Butler et al. 2011).
            Hide
            mgraham Melissa Graham added a comment -

            There's no way you could have known this but, FYI: the attached ascii file MLG_email_TVS_VarChar.txt has a list of everywhere I could find that DM documentation has content related to variability parameters. 

            Show
            mgraham Melissa Graham added a comment - There's no way you could have known this but, FYI: the attached ascii file MLG_email_TVS_VarChar.txt has a list of everywhere I could find that DM documentation has content related to variability parameters. 

              People

              • Assignee:
                ebellm Eric Bellm
                Reporter:
                ebellm Eric Bellm
                Watchers:
                Eric Bellm, Leanne Guy, Melissa Graham
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Due:
                  Created:
                  Updated:

                  Summary Panel