Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11940

Distinguish measurements made on different bands in squash

    XMLWordPrintable

Details

    • Story
    • Status: Done
    • Resolution: Done
    • None
    • squash
    • None

    Description

      It would be nice to display all the bands on one plot by disambiguating by color and linestyle. We also have to make sure we display the specifications correctly since some of them also depend on the band.

      Attachments

        Issue Links

          Activity

            afausti Angelo Fausti added a comment - - edited

            lsst.verify adds filter_name into the job metadata, this information goes to the job.meta field in the SQuaSH DB which is JSON datatype.

            For the verification jobs prior to lsst.verify we have to add the filter_name key in this field by hand. 

            Here are the queries used. For the validation_data_cfht:

            MySQL [squash]> UPDATE job SET meta=JSON_SET(meta, "$.filter_name", "r") WHERE job.ci_dataset='validation_data_cfht';
            
            

            and for the validation_data_hsc. Note the constraint on job.id making sure we don't overwrite the recent jobs where the filter_name is set by lsst.verify:

            MySQL [squash]> UPDATE job SET meta=JSON_SET(meta, "$.filter_name", "HSC-I") WHERE job.ci_dataset='validation_data_hsc' AND job.id < '1240';
            
            

            afausti Angelo Fausti added a comment - - edited lsst.verify adds filter_name  into the job metadata, this information goes to the job.meta  field in the SQuaSH DB which is JSON datatype. For the verification jobs prior to lsst.verify we have to add the filter_name key in this field by hand.  Here are the queries used. For the validation_data_cfht : MySQL [squash]> UPDATE job SET meta=JSON_SET(meta, "$.filter_name", "r") WHERE job.ci_dataset='validation_data_cfht'; and for the validation_data_hsc . Note the constraint on job.id making sure we don't overwrite the recent jobs where the filter_name is set by lsst.verify : MySQL [squash]> UPDATE job SET meta=JSON_SET(meta, "$.filter_name", "HSC-I") WHERE job.ci_dataset='validation_data_hsc' AND job.id < '1240';
            afausti Angelo Fausti added a comment - - edited

            The filter_name attribute is now returned by the SQuaSH RESTful API

            https://squash-restful-api-demo.lsst.codes/monitor

            The filters are color coded using LSST convention and both filter_name and colors are added to the bokeh column datasource. See the get_filter_color() method in squash-bokeh/app/code_changes/base.py

            afausti Angelo Fausti added a comment - - edited The filter_name attribute is now returned by the SQuaSH RESTful API https://squash-restful-api-demo.lsst.codes/monitor The filters are color coded using LSST convention and both filter_name and colors are added to the bokeh column datasource. See the get_filter_color() method in squash-bokeh/app/code_changes/base.py
            afausti Angelo Fausti added a comment - - edited

            The bokeh column datasource has the filter_name attribute and the simplest approach would be to distinguish the data points based on that, something like:

             
            self.plot.circle(x='time', y='value', color='color', legend='filter_name', source=self.cds, fill_color='white', size=12)
            
            

            One issue is that the interactive legend does not work as expected with bokeh 0.12.15

            https://bokeh.pydata.org/en/latest/docs/user_guide/interaction/legends.html

            I decided to leave it like this and wait for the bokeh fix. My expectation is that the interactive legend will let us to select a particular filter and bokeh will set appropriate ranges for the x and y axis.

            The second issue is that the line glyph will not connect the circle glyphs as expected when working with a single column datasource (the line glyph will connect the circle glyphs in the sequence the data is stored)

            One solution for that is to use column datasource views and group the data by filter_name then, in principle, we could connect the circle glyphs with a different line for each filter.

            https://bokeh.pydata.org/en/latest/docs/user_guide/data.html#groupfilter

            afausti Angelo Fausti added a comment - - edited The bokeh column datasource has the filter_name attribute and the simplest approach would be to distinguish the data points based on that, something like:   self.plot.circle(x='time', y='value', color='color', legend='filter_name', source=self.cds, fill_color='white', size=12) One issue is that the interactive legend does not work as expected with bokeh 0.12.15 https://bokeh.pydata.org/en/latest/docs/user_guide/interaction/legends.html I decided to leave it like this and wait for the bokeh fix. My expectation is that the interactive legend will let us to select a particular filter and bokeh will set appropriate ranges for the x and y axis. The second issue is that the line glyph will not connect the circle glyphs as expected when working with a single column datasource (the line glyph will connect the circle glyphs in the sequence the data is stored) One solution for that is to use column datasource views and group the data by filter_name then, in principle, we could connect the circle glyphs with a different line for each filter. https://bokeh.pydata.org/en/latest/docs/user_guide/data.html#groupfilter

             # Attempt to plot a different line for each filter_name
             
                    filters = set(self.cds.data['filter_name'])
                    for filter in filters:
                        view = CDSView(source=self.cds, 
                                       filters=[GroupFilter(column_name='filter_name', 
                                                            group=filter)])
                        
                        self.plot.line(x='time', y='value', color=self.get_filter_color(filter),
                                       source=self.cds, view=view, legend=filter)
            

            afausti Angelo Fausti added a comment - # Attempt to plot a different line for each filter_name   filters = set(self.cds.data['filter_name']) for filter in filters: view = CDSView(source=self.cds, filters=[GroupFilter(column_name='filter_name', group=filter)]) self.plot.line(x='time', y='value', color=self.get_filter_color(filter), source=self.cds, view=view, legend=filter)

            After trying that without success I decided to remove the connecting lines for now.

            afausti Angelo Fausti added a comment - After trying that without success I decided to remove the connecting lines for now.

            See PRs:
            https://github.com/lsst-sqre/squash-restful-api/pull/22
            https://github.com/lsst-sqre/squash-bokeh/pull/17

            krughoff given the issues above, please let me know if you agree with the current solution or if I should investigate more:

            https://squash-demo.lsst.codes/dash/code_changes/

            afausti Angelo Fausti added a comment - See PRs: https://github.com/lsst-sqre/squash-restful-api/pull/22 https://github.com/lsst-sqre/squash-bokeh/pull/17 krughoff given the issues above, please let me know if you agree with the current solution or if I should investigate more: https://squash-demo.lsst.codes/dash/code_changes/

            I guess it's fine to remove the connecting line for now. The grid should help with that a little, though it is very faint on my display. Please file a ticket to investigate re-adding the connecting line.

            krughoff Simon Krughoff (Inactive) added a comment - I guess it's fine to remove the connecting line for now. The grid should help with that a little, though it is very faint on my display. Please file a ticket to investigate re-adding the connecting line.

            Thanks for the review Simon. The production instance as updated accordingly

            https://squash.lsst.codes

            Closing ticket.

            afausti Angelo Fausti added a comment - Thanks for the review Simon. The production instance as updated accordingly https://squash.lsst.codes Closing ticket.

            People

              afausti Angelo Fausti
              afausti Angelo Fausti
              Simon Krughoff (Inactive)
              Angelo Fausti, Simon Krughoff (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.