Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11940

Distinguish measurements made on different bands in squash

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: squash
    • Labels:
      None

      Description

      It would be nice to display all the bands on one plot by disambiguating by color and linestyle. We also have to make sure we display the specifications correctly since some of them also depend on the band.

        Attachments

          Issue Links

            Activity

            Hide
            afausti Angelo Fausti added a comment - - edited

            lsst.verify adds filter_name into the job metadata, this information goes to the job.meta field in the SQuaSH DB which is JSON datatype.

            For the verification jobs prior to lsst.verify we have to add the filter_name key in this field by hand. 

            Here are the queries used. For the validation_data_cfht:

            MySQL [squash]> UPDATE job SET meta=JSON_SET(meta, "$.filter_name", "r") WHERE job.ci_dataset='validation_data_cfht';
            
            

            and for the validation_data_hsc. Note the constraint on job.id making sure we don't overwrite the recent jobs where the filter_name is set by lsst.verify:

            MySQL [squash]> UPDATE job SET meta=JSON_SET(meta, "$.filter_name", "HSC-I") WHERE job.ci_dataset='validation_data_hsc' AND job.id < '1240';
            
            

            Show
            afausti Angelo Fausti added a comment - - edited lsst.verify adds filter_name  into the job metadata, this information goes to the job.meta  field in the SQuaSH DB which is JSON datatype. For the verification jobs prior to lsst.verify we have to add the filter_name key in this field by hand.  Here are the queries used. For the validation_data_cfht : MySQL [squash]> UPDATE job SET meta=JSON_SET(meta, "$.filter_name", "r") WHERE job.ci_dataset='validation_data_cfht'; and for the validation_data_hsc . Note the constraint on job.id making sure we don't overwrite the recent jobs where the filter_name is set by lsst.verify : MySQL [squash]> UPDATE job SET meta=JSON_SET(meta, "$.filter_name", "HSC-I") WHERE job.ci_dataset='validation_data_hsc' AND job.id < '1240';
            Hide
            afausti Angelo Fausti added a comment - - edited

            The filter_name attribute is now returned by the SQuaSH RESTful API

            https://squash-restful-api-demo.lsst.codes/monitor

            The filters are color coded using LSST convention and both filter_name and colors are added to the bokeh column datasource. See the get_filter_color() method in squash-bokeh/app/code_changes/base.py

            Show
            afausti Angelo Fausti added a comment - - edited The filter_name attribute is now returned by the SQuaSH RESTful API https://squash-restful-api-demo.lsst.codes/monitor The filters are color coded using LSST convention and both filter_name and colors are added to the bokeh column datasource. See the get_filter_color() method in squash-bokeh/app/code_changes/base.py
            Hide
            afausti Angelo Fausti added a comment - - edited

            The bokeh column datasource has the filter_name attribute and the simplest approach would be to distinguish the data points based on that, something like:

             
            self.plot.circle(x='time', y='value', color='color', legend='filter_name', source=self.cds, fill_color='white', size=12)
            
            

            One issue is that the interactive legend does not work as expected with bokeh 0.12.15

            https://bokeh.pydata.org/en/latest/docs/user_guide/interaction/legends.html

            I decided to leave it like this and wait for the bokeh fix. My expectation is that the interactive legend will let us to select a particular filter and bokeh will set appropriate ranges for the x and y axis.

            The second issue is that the line glyph will not connect the circle glyphs as expected when working with a single column datasource (the line glyph will connect the circle glyphs in the sequence the data is stored)

            One solution for that is to use column datasource views and group the data by filter_name then, in principle, we could connect the circle glyphs with a different line for each filter.

            https://bokeh.pydata.org/en/latest/docs/user_guide/data.html#groupfilter

            Show
            afausti Angelo Fausti added a comment - - edited The bokeh column datasource has the filter_name attribute and the simplest approach would be to distinguish the data points based on that, something like:   self.plot.circle(x='time', y='value', color='color', legend='filter_name', source=self.cds, fill_color='white', size=12) One issue is that the interactive legend does not work as expected with bokeh 0.12.15 https://bokeh.pydata.org/en/latest/docs/user_guide/interaction/legends.html I decided to leave it like this and wait for the bokeh fix. My expectation is that the interactive legend will let us to select a particular filter and bokeh will set appropriate ranges for the x and y axis. The second issue is that the line glyph will not connect the circle glyphs as expected when working with a single column datasource (the line glyph will connect the circle glyphs in the sequence the data is stored) One solution for that is to use column datasource views and group the data by filter_name then, in principle, we could connect the circle glyphs with a different line for each filter. https://bokeh.pydata.org/en/latest/docs/user_guide/data.html#groupfilter
            Hide
            afausti Angelo Fausti added a comment -

             # Attempt to plot a different line for each filter_name
             
                    filters = set(self.cds.data['filter_name'])
                    for filter in filters:
                        view = CDSView(source=self.cds, 
                                       filters=[GroupFilter(column_name='filter_name', 
                                                            group=filter)])
                        
                        self.plot.line(x='time', y='value', color=self.get_filter_color(filter),
                                       source=self.cds, view=view, legend=filter)
            

            Show
            afausti Angelo Fausti added a comment - # Attempt to plot a different line for each filter_name   filters = set(self.cds.data['filter_name']) for filter in filters: view = CDSView(source=self.cds, filters=[GroupFilter(column_name='filter_name', group=filter)]) self.plot.line(x='time', y='value', color=self.get_filter_color(filter), source=self.cds, view=view, legend=filter)
            Hide
            afausti Angelo Fausti added a comment -

            After trying that without success I decided to remove the connecting lines for now.

            Show
            afausti Angelo Fausti added a comment - After trying that without success I decided to remove the connecting lines for now.
            Hide
            afausti Angelo Fausti added a comment -

            See PRs:
            https://github.com/lsst-sqre/squash-restful-api/pull/22
            https://github.com/lsst-sqre/squash-bokeh/pull/17

            Simon Krughoff given the issues above, please let me know if you agree with the current solution or if I should investigate more:

            https://squash-demo.lsst.codes/dash/code_changes/

            Show
            afausti Angelo Fausti added a comment - See PRs: https://github.com/lsst-sqre/squash-restful-api/pull/22 https://github.com/lsst-sqre/squash-bokeh/pull/17 Simon Krughoff given the issues above, please let me know if you agree with the current solution or if I should investigate more: https://squash-demo.lsst.codes/dash/code_changes/
            Hide
            krughoff Simon Krughoff added a comment -

            I guess it's fine to remove the connecting line for now. The grid should help with that a little, though it is very faint on my display. Please file a ticket to investigate re-adding the connecting line.

            Show
            krughoff Simon Krughoff added a comment - I guess it's fine to remove the connecting line for now. The grid should help with that a little, though it is very faint on my display. Please file a ticket to investigate re-adding the connecting line.
            Hide
            afausti Angelo Fausti added a comment -

            Thanks for the review Simon. The production instance as updated accordingly

            https://squash.lsst.codes

            Closing ticket.

            Show
            afausti Angelo Fausti added a comment - Thanks for the review Simon. The production instance as updated accordingly https://squash.lsst.codes Closing ticket.

              People

              Assignee:
              afausti Angelo Fausti
              Reporter:
              afausti Angelo Fausti
              Reviewers:
              Simon Krughoff
              Watchers:
              Angelo Fausti, Simon Krughoff
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.