Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15554

Convert pipe_tasks to numpydoc status

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Invalid
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: pipe_tasks
    • Labels:
    • Sprint:
      AP F18-4, AP F18-5, AP F18-6, AP S19-1, AP F22-3 (August)
    • Team:
      Alert Production

      Attachments

        Issue Links

          Activity

          No builds found.
          dstaker David Staker [X] (Inactive) created issue -
          dstaker David Staker [X] (Inactive) made changes -
          Field Original Value New Value
          Risk Score 0
          swinbank John Swinbank made changes -
          Epic Link DM-14447 [ 80385 ]
          swinbank John Swinbank made changes -
          Sprint AP F18-4 [ 749 ]
          Team Alert Production [ 10300 ]
          dstaker David Staker [X] (Inactive) made changes -
          Assignee David Staker [ dstaker ]
          dstaker David Staker [X] (Inactive) made changes -
          Watchers David Staker [ David Staker ] David Staker, Meredith Rawls [ David Staker, Meredith Rawls ]
          Reviewers Meredith Rawls [ mrawls ]
          Hide
          mrawls Meredith Rawls added a comment -

          When you get a chance, David, please put this ticket as "In Review" (under workflow) and update your GitHub PR so the title is "DM-15554: Convert pipe_tasks to numpydoc status" (to match the name of the ticket). I'll take a look next week!

          Show
          mrawls Meredith Rawls added a comment - When you get a chance, David, please put this ticket as "In Review" (under workflow) and update your GitHub PR so the title is " DM-15554 : Convert pipe_tasks to numpydoc status" (to match the name of the ticket). I'll take a look next week!
          dstaker David Staker [X] (Inactive) made changes -
          Status To Do [ 10001 ] In Review [ 10004 ]
          swinbank John Swinbank made changes -
          Remote Link This issue links to "Page (Confluence)" [ 17965 ]
          swinbank John Swinbank made changes -
          Sprint AP F18-4 [ 749 ] AP F18-4, AP F18-5 [ 749, 750 ]
          swinbank John Swinbank made changes -
          Sprint AP F18-4, AP F18-5 [ 749, 750 ] AP F18-4, AP F18-5, AP F18-6 [ 749, 750, 751 ]
          swinbank John Swinbank made changes -
          Assignee David Staker [ dstaker ] John Swinbank [ swinbank ]
          swinbank John Swinbank made changes -
          Epic Link DM-14447 [ 80385 ] DM-16722 [ 235355 ]
          swinbank John Swinbank made changes -
          Sprint AP F18-4, AP F18-5, AP F18-6 [ 749, 750, 751 ] AP F18-4, AP F18-5, AP F18-6, AP S19-1 [ 749, 750, 751, 825 ]
          swinbank John Swinbank made changes -
          Assignee John Swinbank [ swinbank ]
          swinbank John Swinbank made changes -
          Status In Review [ 10004 ] In Progress [ 3 ]
          krzys Krzysztof Findeisen made changes -
          Labels numpydoc
          gkovacs Gabor Kovacs [X] (Inactive) made changes -
          Assignee Gabor Kovacs [ gkovacs ]
          gkovacs Gabor Kovacs [X] (Inactive) made changes -
          Hide
          gkovacs Gabor Kovacs [X] (Inactive) added a comment - - edited

          DM-15554 was left in an inconsistent state. Commits were made in branch “15554” and “tickets/DM-15554” which were merged on tickets/DM-15554 without properly resolving the conflicts. (The merge commit contained all the conflict markers in the files.)

          At today’s hacking (2018-12-17), I removed the DM-15554 merge commit, cherry-picked the two commits from branch “15554” instead and then rebased the branch to master. Unfortunately, the before the hackday state of "tickets/DM-15554" was not saved, the before rebase version is saved as u/gkovacs/hack2018-12-17_DM-15554.

          I tried to keep code from the master and converted documentation from the branch, though there were some minor parallel evolution. There is an additional manual commit for minor changes. Notes:

          • exampleCmdLineTask.py - old style documentation is now commented out
          • _init_ documentation is removed where the class documentation is already updated.
          • _all_ was set in the branch in the meantime in the following files: imageDifference.py ingest.py ingestCalibs.py ingestPgsql.py makeDiscreteSkyMap.py makeSkyMap.py matchBackgrounds.py measurePsf.py objectMasks.py repair.py scaleVariance.py setPrimaryFlags.py .
          • multiBand.py — removed import numpy that was added as the new style docstring block
          • objectMasks.py - readFits method docstring is somewhat converted but commented out, needs further manual consideration
          • scaleZeroPoint.py - kept docstring in branch _all_ which is more extended than the master version
          • setConfigFromEups.py — In the master version setAstrometryConfigFromEups() docstring was already modified in the old format, added the wildcards (asterisk).
          • snapCombine.py transformMeasurement.py - Docstring brief should start with opening triple quotes line.

          Package trial build testing and docs build failed with the same error at end of day, see attachment; this may be something inconsistent with the docs build on my desktop now, unrelated to the changes in this branch now.

           

          Sorry, I could not check whether the new docstrings look OK.

           

          Please also see CI build whether code is not got trivially broken.

          Show
          gkovacs Gabor Kovacs [X] (Inactive) added a comment - - edited DM-15554 was left in an inconsistent state. Commits were made in branch “15554” and “tickets/ DM-15554 ” which were merged on tickets/ DM-15554 without properly resolving the conflicts. (The merge commit contained all the conflict markers in the files.) At today’s hacking (2018-12-17), I removed the DM-15554 merge commit, cherry-picked the two commits from branch “15554” instead and then rebased the branch to master. Unfortunately, the before the hackday state of "tickets/ DM-15554 " was not saved, the before rebase version is saved as u/gkovacs/hack2018-12-17_ DM-15554 . I tried to keep code from the master and converted documentation from the branch, though there were some minor parallel evolution. There is an additional manual commit for minor changes. Notes: exampleCmdLineTask.py - old style documentation is now commented out _ init _ documentation is removed where the class documentation is already updated. _ all _ was set in the branch in the meantime in the following files: imageDifference.py ingest.py ingestCalibs.py ingestPgsql.py makeDiscreteSkyMap.py makeSkyMap.py matchBackgrounds.py measurePsf.py objectMasks.py repair.py scaleVariance.py setPrimaryFlags.py . multiBand.py — removed import numpy that was added as the new style docstring block objectMasks.py - readFits method docstring is somewhat converted but commented out, needs further manual consideration scaleZeroPoint.py - kept docstring in branch _ all _ which is more extended than the master version setConfigFromEups.py — In the master version setAstrometryConfigFromEups() docstring was already modified in the old format, added the wildcards (asterisk). snapCombine.py transformMeasurement.py - Docstring brief should start with opening triple quotes line. Package trial build testing and docs build failed with the same error at end of day, see attachment; this may be something inconsistent with the docs build on my desktop now, unrelated to the changes in this branch now.   Sorry, I could not check whether the new docstrings look OK.   Please also see CI build whether code is not got trivially broken.
          gkovacs Gabor Kovacs [X] (Inactive) made changes -
          Assignee Gabor Kovacs [ gkovacs ]
          mrawls Meredith Rawls made changes -
          Reviewers Meredith Rawls [ mrawls ]
          Assignee Meredith Rawls [ mrawls ]
          Hide
          mrawls Meredith Rawls added a comment -

          I completed assembleCoadd.py and calibrate.py today. The former was mostly already done (thanks past me and DM-13950), but the large swaths of text now live in the new task topic type docs. The latter took a lot of hunting to figure out what various parameter types were.

          Show
          mrawls Meredith Rawls added a comment - I completed assembleCoadd.py and calibrate.py today. The former was mostly already done (thanks past me and DM-13950 ), but the large swaths of text now live in the new task topic type docs. The latter took a lot of hunting to figure out what various parameter types were.
          Hide
          mrawls Meredith Rawls added a comment -

          I completed characterizeImage.py, measurePsf.py, and repair.py today, along with their corresponding task topic docs.

          Show
          mrawls Meredith Rawls added a comment - I completed characterizeImage.py, measurePsf.py, and repair.py today, along with their corresponding task topic docs.
          Hide
          mrawls Meredith Rawls added a comment -

          I'm not going to get any more done on this for now, so I'm un-assigning it from myself.

          Brief current status is as follows:

          • Initial numpydoc conversion has happened in all .py files, and skeleton task topic doc files exist
          • assembleCoadd.py, calibrate.py, characterizeImage.py, measurePsf.py, and repair.py are all nominally complete, as well as their corresponding task topic docs
          • The other 37 python files still need to be gone through manually. The bulk of the work will involve looking up the correct object type for various parameters, moving debug/example docstring prose to the task topic docs, making sure all input/return variables are included in all task/function docstrings, and removing and condensing duplicate/contradictory descriptions of tasks/functions.
          • Building the docs and fixing any remaining warnings that pertain to pipe_tasks
          • Building the package/running Jenkins to ensure nothing is broken

          All of the remaining work is easily 10 story points or more. It may make sense to break this up into a few sub-tickets so less time is spent dealing with rebasing. At the time of this writing, the ticket branch is up to date with master.

          Show
          mrawls Meredith Rawls added a comment - I'm not going to get any more done on this for now, so I'm un-assigning it from myself. Brief current status is as follows: Initial numpydoc conversion has happened in all .py files, and skeleton task topic doc files exist assembleCoadd.py, calibrate.py, characterizeImage.py, measurePsf.py, and repair.py are all nominally complete, as well as their corresponding task topic docs The other 37 python files still need to be gone through manually. The bulk of the work will involve looking up the correct object type for various parameters, moving debug/example docstring prose to the task topic docs, making sure all input/return variables are included in all task/function docstrings, and removing and condensing duplicate/contradictory descriptions of tasks/functions. Building the docs and fixing any remaining warnings that pertain to pipe_tasks Building the package/running Jenkins to ensure nothing is broken All of the remaining work is easily 10 story points or more. It may make sense to break this up into a few sub-tickets so less time is spent dealing with rebasing. At the time of this writing, the ticket branch is up to date with master.
          mrawls Meredith Rawls made changes -
          Watchers David Staker, Gabor Kovacs, Meredith Rawls [ David Staker, Gabor Kovacs, Meredith Rawls ] David Staker, Gabor Kovacs, John Swinbank, Meredith Rawls [ David Staker, Gabor Kovacs, John Swinbank, Meredith Rawls ]
          mrawls Meredith Rawls made changes -
          Assignee Meredith Rawls [ mrawls ]
          swinbank John Swinbank made changes -
          Sprint AP F18-4, AP F18-5, AP F18-6, AP S19-1 [ 749, 750, 751, 825 ] AP F18-4, AP F18-5, AP F18-6, AP S19-1, AP S19-2 [ 749, 750, 751, 825, 830 ]
          swinbank John Swinbank made changes -
          Sprint AP F18-4, AP F18-5, AP F18-6, AP S19-1, AP S19-2 [ 749, 750, 751, 825, 830 ] AP F18-4, AP F18-5, AP F18-6, AP S19-1 [ 749, 750, 751, 825 ]
          frossie Frossie Economou made changes -
          Status Admin Review [ 3 ] In Progress [ 11605 ]
          frossie Frossie Economou made changes -
          Status Review [ 11605 ] In Progress [ 3 ]
          swinbank John Swinbank made changes -
          Epic Link DM-16722 [ 235355 ] DM-17887 [ 240317 ]
          swinbank John Swinbank made changes -
          Epic Link DM-17887 [ 240317 ] DM-19979 [ 307530 ]
          swinbank John Swinbank made changes -
          Status In Progress [ 3 ] To Do [ 10001 ]
          swinbank John Swinbank made changes -
          Epic Link DM-19979 [ 307530 ] DM-21441 [ 423048 ]
          swinbank John Swinbank made changes -
          Link This issue is duplicated by DM-22464 [ DM-22464 ]
          swinbank John Swinbank made changes -
          Epic Link DM-21441 [ 423048 ] DM-22484 [ 427311 ]
          swinbank John Swinbank made changes -
          Epic Link DM-22484 [ 427311 ] DM-24339 [ 433026 ]
          swinbank John Swinbank made changes -
          Epic Link DM-24339 [ 433026 ] DM-25139 [ 435257 ]
          swinbank John Swinbank made changes -
          Epic Link DM-25139 [ 435257 ] DM-26810 [ 439762 ]
          sullivan Ian Sullivan made changes -
          Epic Link DM-26810 [ 439762 ] DM-27906 [ 442554 ]
          sullivan Ian Sullivan made changes -
          Epic Link DM-27906 [ 442554 ] DM-29214 [ 459218 ]
          sullivan Ian Sullivan made changes -
          Epic Link DM-29214 [ 459218 ] DM-30436 [ 504824 ]
          sullivan Ian Sullivan made changes -
          Epic Link DM-30436 [ 504824 ] DM-30501 [ 510159 ]
          sullivan Ian Sullivan made changes -
          Epic Link DM-30501 [ 510159 ] DM-30502 [ 510160 ]
          sullivan Ian Sullivan made changes -
          Epic Link DM-30502 [ 510160 ] DM-30506 [ 510172 ]
          horvat Nikolina Horvat made changes -
          Epic Link DM-30506 [ 510172 ] DM-34931 [ 1598492 ]
          tjenness Tim Jenness made changes -
          Component/s pipe_tasks [ 10726 ]
          Parejkoj John Parejko made changes -
          Assignee Erin Howard [ JIRAUSER20420 ]
          Parejkoj John Parejko made changes -
          Link This issue is triggering DM-35939 [ DM-35939 ]
          Parejkoj John Parejko made changes -
          Link This issue relates to DM-35940 [ DM-35940 ]
          sullivan Ian Sullivan made changes -
          Sprint AP F18-4, AP F18-5, AP F18-6, AP S19-1 [ 749, 750, 751, 825 ] AP F18-4, AP F18-5, AP F18-6, AP S19-1, AP F22-3 (August) [ 749, 750, 751, 825, 1176 ]
          sullivan Ian Sullivan made changes -
          Rank Ranked lower
          howarde7 Erin Howard made changes -
          Status To Do [ 10001 ] In Progress [ 3 ]
          sullivan Ian Sullivan made changes -
          Status In Progress [ 3 ] To Do [ 10001 ]
          sullivan Ian Sullivan made changes -
          Resolution Done [ 10000 ]
          Status To Do [ 10001 ] Invalid [ 11005 ]
          sullivan Ian Sullivan made changes -
          Link This issue is triggering DM-35939 [ DM-35939 ]
          Hide
          sullivan Ian Sullivan added a comment -

          All of the final work and ticket branches will be merged from DM-35939.

          Show
          sullivan Ian Sullivan added a comment - All of the final work and ticket branches will be merged from DM-35939 .
          sullivan Ian Sullivan made changes -
          Link This issue is duplicated by DM-35939 [ DM-35939 ]

            People

            Assignee:
            howarde7 Erin Howard
            Reporter:
            dstaker David Staker [X] (Inactive)
            Watchers:
            David Staker [X] (Inactive), Gabor Kovacs [X] (Inactive), Ian Sullivan, John Swinbank, Meredith Rawls
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.