Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15554

Convert pipe_tasks to numpydoc status

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Invalid
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: pipe_tasks
    • Labels:
    • Sprint:
      AP F18-4, AP F18-5, AP F18-6, AP S19-1, AP F22-3 (August)
    • Team:
      Alert Production

      Attachments

        Issue Links

          Activity

          Hide
          mrawls Meredith Rawls added a comment -

          When you get a chance, David, please put this ticket as "In Review" (under workflow) and update your GitHub PR so the title is "DM-15554: Convert pipe_tasks to numpydoc status" (to match the name of the ticket). I'll take a look next week!

          Show
          mrawls Meredith Rawls added a comment - When you get a chance, David, please put this ticket as "In Review" (under workflow) and update your GitHub PR so the title is " DM-15554 : Convert pipe_tasks to numpydoc status" (to match the name of the ticket). I'll take a look next week!
          Hide
          gkovacs Gabor Kovacs [X] (Inactive) added a comment - - edited

          DM-15554 was left in an inconsistent state. Commits were made in branch “15554” and “tickets/DM-15554” which were merged on tickets/DM-15554 without properly resolving the conflicts. (The merge commit contained all the conflict markers in the files.)

          At today’s hacking (2018-12-17), I removed the DM-15554 merge commit, cherry-picked the two commits from branch “15554” instead and then rebased the branch to master. Unfortunately, the before the hackday state of "tickets/DM-15554" was not saved, the before rebase version is saved as u/gkovacs/hack2018-12-17_DM-15554.

          I tried to keep code from the master and converted documentation from the branch, though there were some minor parallel evolution. There is an additional manual commit for minor changes. Notes:

          • exampleCmdLineTask.py - old style documentation is now commented out
          • _init_ documentation is removed where the class documentation is already updated.
          • _all_ was set in the branch in the meantime in the following files: imageDifference.py ingest.py ingestCalibs.py ingestPgsql.py makeDiscreteSkyMap.py makeSkyMap.py matchBackgrounds.py measurePsf.py objectMasks.py repair.py scaleVariance.py setPrimaryFlags.py .
          • multiBand.py — removed import numpy that was added as the new style docstring block
          • objectMasks.py - readFits method docstring is somewhat converted but commented out, needs further manual consideration
          • scaleZeroPoint.py - kept docstring in branch _all_ which is more extended than the master version
          • setConfigFromEups.py — In the master version setAstrometryConfigFromEups() docstring was already modified in the old format, added the wildcards (asterisk).
          • snapCombine.py transformMeasurement.py - Docstring brief should start with opening triple quotes line.

          Package trial build testing and docs build failed with the same error at end of day, see attachment; this may be something inconsistent with the docs build on my desktop now, unrelated to the changes in this branch now.

           

          Sorry, I could not check whether the new docstrings look OK.

           

          Please also see CI build whether code is not got trivially broken.

          Show
          gkovacs Gabor Kovacs [X] (Inactive) added a comment - - edited DM-15554 was left in an inconsistent state. Commits were made in branch “15554” and “tickets/ DM-15554 ” which were merged on tickets/ DM-15554 without properly resolving the conflicts. (The merge commit contained all the conflict markers in the files.) At today’s hacking (2018-12-17), I removed the DM-15554 merge commit, cherry-picked the two commits from branch “15554” instead and then rebased the branch to master. Unfortunately, the before the hackday state of "tickets/ DM-15554 " was not saved, the before rebase version is saved as u/gkovacs/hack2018-12-17_ DM-15554 . I tried to keep code from the master and converted documentation from the branch, though there were some minor parallel evolution. There is an additional manual commit for minor changes. Notes: exampleCmdLineTask.py - old style documentation is now commented out _ init _ documentation is removed where the class documentation is already updated. _ all _ was set in the branch in the meantime in the following files: imageDifference.py ingest.py ingestCalibs.py ingestPgsql.py makeDiscreteSkyMap.py makeSkyMap.py matchBackgrounds.py measurePsf.py objectMasks.py repair.py scaleVariance.py setPrimaryFlags.py . multiBand.py — removed import numpy that was added as the new style docstring block objectMasks.py - readFits method docstring is somewhat converted but commented out, needs further manual consideration scaleZeroPoint.py - kept docstring in branch _ all _ which is more extended than the master version setConfigFromEups.py — In the master version setAstrometryConfigFromEups() docstring was already modified in the old format, added the wildcards (asterisk). snapCombine.py transformMeasurement.py - Docstring brief should start with opening triple quotes line. Package trial build testing and docs build failed with the same error at end of day, see attachment; this may be something inconsistent with the docs build on my desktop now, unrelated to the changes in this branch now.   Sorry, I could not check whether the new docstrings look OK.   Please also see CI build whether code is not got trivially broken.
          Hide
          mrawls Meredith Rawls added a comment -

          I completed assembleCoadd.py and calibrate.py today. The former was mostly already done (thanks past me and DM-13950), but the large swaths of text now live in the new task topic type docs. The latter took a lot of hunting to figure out what various parameter types were.

          Show
          mrawls Meredith Rawls added a comment - I completed assembleCoadd.py and calibrate.py today. The former was mostly already done (thanks past me and DM-13950 ), but the large swaths of text now live in the new task topic type docs. The latter took a lot of hunting to figure out what various parameter types were.
          Hide
          mrawls Meredith Rawls added a comment -

          I completed characterizeImage.py, measurePsf.py, and repair.py today, along with their corresponding task topic docs.

          Show
          mrawls Meredith Rawls added a comment - I completed characterizeImage.py, measurePsf.py, and repair.py today, along with their corresponding task topic docs.
          Hide
          mrawls Meredith Rawls added a comment -

          I'm not going to get any more done on this for now, so I'm un-assigning it from myself.

          Brief current status is as follows:

          • Initial numpydoc conversion has happened in all .py files, and skeleton task topic doc files exist
          • assembleCoadd.py, calibrate.py, characterizeImage.py, measurePsf.py, and repair.py are all nominally complete, as well as their corresponding task topic docs
          • The other 37 python files still need to be gone through manually. The bulk of the work will involve looking up the correct object type for various parameters, moving debug/example docstring prose to the task topic docs, making sure all input/return variables are included in all task/function docstrings, and removing and condensing duplicate/contradictory descriptions of tasks/functions.
          • Building the docs and fixing any remaining warnings that pertain to pipe_tasks
          • Building the package/running Jenkins to ensure nothing is broken

          All of the remaining work is easily 10 story points or more. It may make sense to break this up into a few sub-tickets so less time is spent dealing with rebasing. At the time of this writing, the ticket branch is up to date with master.

          Show
          mrawls Meredith Rawls added a comment - I'm not going to get any more done on this for now, so I'm un-assigning it from myself. Brief current status is as follows: Initial numpydoc conversion has happened in all .py files, and skeleton task topic doc files exist assembleCoadd.py, calibrate.py, characterizeImage.py, measurePsf.py, and repair.py are all nominally complete, as well as their corresponding task topic docs The other 37 python files still need to be gone through manually. The bulk of the work will involve looking up the correct object type for various parameters, moving debug/example docstring prose to the task topic docs, making sure all input/return variables are included in all task/function docstrings, and removing and condensing duplicate/contradictory descriptions of tasks/functions. Building the docs and fixing any remaining warnings that pertain to pipe_tasks Building the package/running Jenkins to ensure nothing is broken All of the remaining work is easily 10 story points or more. It may make sense to break this up into a few sub-tickets so less time is spent dealing with rebasing. At the time of this writing, the ticket branch is up to date with master.
          Hide
          sullivan Ian Sullivan added a comment -

          All of the final work and ticket branches will be merged from DM-35939.

          Show
          sullivan Ian Sullivan added a comment - All of the final work and ticket branches will be merged from DM-35939 .

            People

            Assignee:
            howarde7 Erin Howard
            Reporter:
            dstaker David Staker [X] (Inactive)
            Watchers:
            David Staker [X] (Inactive), Gabor Kovacs [X] (Inactive), Ian Sullivan, John Swinbank, Meredith Rawls
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.