Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-472

Create an iPython Notebook visualization of the LSST Demo Data

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Story Points:
      4
    • Team:
      SQuaRE

      Description

      Develop an iPython Notebook to illustrate the processing and results of the LSST Demo. This can be used as both a basic tutorial for the Stack, and a how-to for creating additional visualizations for DM. The visualization should include the source catalog, and possibly also one or more of the processed images.

        Attachments

          Activity

          Hide
          shaw Richard Shaw [X] (Inactive) added a comment -

          Russell, would you have time to review a (small) iPython notebook I put together for the LSST Stack Demo? I attached it to the Testing the Installation page on Confluence: https://confluence.lsstcorp.org/display/LSWUG/Testing+the+Installation

          The easiest thing is to clone the lsst_dm_stack_demo, run the demo, plop the StackDemo.ipynb in that directory, follow the setup instructions, and invoke it with "ipython notebook".

          I should mention that I am not that fluent with notebooks, so I'd appreciate any advice you can give that would improve it.

          Show
          shaw Richard Shaw [X] (Inactive) added a comment - Russell, would you have time to review a (small) iPython notebook I put together for the LSST Stack Demo? I attached it to the Testing the Installation page on Confluence: https://confluence.lsstcorp.org/display/LSWUG/Testing+the+Installation The easiest thing is to clone the lsst_dm_stack_demo, run the demo, plop the StackDemo.ipynb in that directory, follow the setup instructions, and invoke it with "ipython notebook". I should mention that I am not that fluent with notebooks, so I'd appreciate any advice you can give that would improve it.
          Hide
          rowen Russell Owen added a comment - - edited

          The page <https://confluence.lsstcorp.org/display/LSWUG/Testing+the+Installation> is mostly quite clear. A few minor nits:

          I find the "Demo Output" table and explanatory text confusing. I suspect that detected-sources.txt.expected and detected-sources_small.txt.expected are always written. In addition, one of detected-sources.txt or detected-sources_small.txt is written, depending on which script is run, and this should match the ".expected" version. In my case it does NOT match, which is disturbing, but I have no idea how serious a problem it is that they do not match.

          afw is already setup by running the demo, so your instructions for visualizing the output do not need to say to do that.

          Please provide or link to instructions for running an ipython notebook. I ran into a lot of trouble getting it going because I had forgotten how to do it and double-clicking the file caused all sorts of problems (which I won't go into, but I'd like to save other Mac users the pain). I think these instructions will work:

          $ conda install ipython-notebook
          $ ipython notebook

          then the page that is displayed in your web browser to run StackDemo.ipynb

          The notebook itself looks very nice to me. A scientist might be able to offer useful tips on enhancing the content, but to me it seems to provide a useful introduction to visualizing data.Here are a few minor nits:

          Consider changing:
          "This visualization builds upon the <LSST Demo>; which assumes you have done the following to load the environment, retrieve the demo and run it:"
          to something like:
          "This visualizes data generated by <running the LSST Demo>"
          If that links to the appropriate section then you can avoid duplicating how to run in this section. Even if you keep the duplication, I think you can lose the line:

          1. Download the StackDemo.ipynb notebook.

          Typo: "Starb" -> Start

          I had to remind myself how to run a cell (control-return on MacOS X), but the built-in help was helpful.

          I see thin green vertical lines in the ds9 image from the second cell. It's hard to believe these are saturated pixels. It would be usful to explain what's going on.

          maskPlaneColors is a cell, but there is no reason to run it, is there? This is slightly confusing, though running it is harmless.

          bug in demoDisplay (in the last cell): I think it will crash due to `frameTitle` being undefined if `title` if not blank.

          Show
          rowen Russell Owen added a comment - - edited The page < https://confluence.lsstcorp.org/display/LSWUG/Testing+the+Installation > is mostly quite clear. A few minor nits: I find the "Demo Output" table and explanatory text confusing. I suspect that detected-sources.txt.expected and detected-sources_small.txt.expected are always written. In addition, one of detected-sources.txt or detected-sources_small.txt is written, depending on which script is run, and this should match the ".expected" version. In my case it does NOT match, which is disturbing, but I have no idea how serious a problem it is that they do not match. afw is already setup by running the demo, so your instructions for visualizing the output do not need to say to do that. Please provide or link to instructions for running an ipython notebook. I ran into a lot of trouble getting it going because I had forgotten how to do it and double-clicking the file caused all sorts of problems (which I won't go into, but I'd like to save other Mac users the pain). I think these instructions will work: $ conda install ipython-notebook $ ipython notebook then the page that is displayed in your web browser to run StackDemo.ipynb The notebook itself looks very nice to me. A scientist might be able to offer useful tips on enhancing the content, but to me it seems to provide a useful introduction to visualizing data.Here are a few minor nits: Consider changing: "This visualization builds upon the <LSST Demo>; which assumes you have done the following to load the environment, retrieve the demo and run it:" to something like: "This visualizes data generated by <running the LSST Demo>" If that links to the appropriate section then you can avoid duplicating how to run in this section. Even if you keep the duplication, I think you can lose the line: Download the StackDemo.ipynb notebook. Typo: "Starb" -> Start I had to remind myself how to run a cell (control-return on MacOS X), but the built-in help was helpful. I see thin green vertical lines in the ds9 image from the second cell. It's hard to believe these are saturated pixels. It would be usful to explain what's going on. maskPlaneColors is a cell, but there is no reason to run it, is there? This is slightly confusing, though running it is harmless. bug in demoDisplay (in the last cell): I think it will crash due to `frameTitle` being undefined if `title` if not blank.
          Hide
          shaw Richard Shaw [X] (Inactive) added a comment -

          Thanks for your review; your suggestions significantly improved the presentation. Basically I adopted all of your suggestions. A few points are perhaps worth mentioning.

          1. I heavily revised the "Fetching & Running the Demo" section. We changed the way that the demo scripts work, and the fallout had not been factored into this page.

          2. I added a link to an introduction to iPython notebooks. I guess I assumed too much about the knowledge that users would have. (I thought I was the only one for which this was new.)

          3. Yes, the demo output doesn't match the expected output. This is a known issue to be resolved after the current crisis with publishing the 9.x release.

          4. Eliminated some of the redundant material in the notebook intro. I kept some because I don't want to assume that the notebook will be run right after the demo generates data.

          5. Fixed the bug in demoDisplay (doh!) Thanks.

          Show
          shaw Richard Shaw [X] (Inactive) added a comment - Thanks for your review; your suggestions significantly improved the presentation. Basically I adopted all of your suggestions. A few points are perhaps worth mentioning. 1. I heavily revised the "Fetching & Running the Demo" section. We changed the way that the demo scripts work, and the fallout had not been factored into this page. 2. I added a link to an introduction to iPython notebooks. I guess I assumed too much about the knowledge that users would have. (I thought I was the only one for which this was new.) 3. Yes, the demo output doesn't match the expected output. This is a known issue to be resolved after the current crisis with publishing the 9.x release. 4. Eliminated some of the redundant material in the notebook intro. I kept some because I don't want to assume that the notebook will be run right after the demo generates data. 5. Fixed the bug in demoDisplay (doh!) Thanks.

            People

            • Assignee:
              shaw Richard Shaw [X] (Inactive)
              Reporter:
              shaw Richard Shaw [X] (Inactive)
              Reviewers:
              Russell Owen
              Watchers:
              Mario Juric, Richard Shaw [X] (Inactive), Russell Owen
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel