Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-83

Rename pipe_tasks' ingest.py

    XMLWordPrintable

Details

    • RFC
    • Status: Retired
    • Resolution: Done
    • DM
    • None
    • Jira

    Description

      The ingestImages.py bin script provides a camera-agnostic manner of creating a data repository (including a registry). The back-end code resides in pipe_tasks under the name ingest.py, and the IngestTask._DefaultName = "ingest", which means that configuration files in obs packages are also named ingest.py. This choice of name was unfortunate, as it may be confused with ingest of sources into the database. DM-3439 proposes changing ingest.py to ingestImages.py to reduce confusion. Suggestions for better names are also welcome.

      This will require coordinated changes to pipe_tasks, obs_subaru, obs_cfht and obs_decam.

      Attachments

        Issue Links

          Activity

            I'd avoid the term ingest entirely — registerImages?

            rhl Robert Lupton added a comment - I'd avoid the term ingest entirely — registerImages?
            rowen Russell Owen added a comment -

            Your suggestion sounds like a big improvement. Nonetheless, I'll ask for even more: I think "ingestImages.py" is too ambiguous: it sounds as if images are being ingested to a database, or at least added to an existing collection of some kind. So I will request a clearer and less ambiguous name.

            I suggest "makeRepoFromImages". It is rather long, but quite clear and unambiguous. If the operation is sufficiently rare then the long name will not hurt, and auto-completion will work once one types "makeRepo".

            "repoFromImages" might do, but I miss the verb. I do not like "imagesToRepo" because it sounds as if it might be adding images to an existing repo.

            rowen Russell Owen added a comment - Your suggestion sounds like a big improvement. Nonetheless, I'll ask for even more: I think "ingestImages.py" is too ambiguous: it sounds as if images are being ingested to a database, or at least added to an existing collection of some kind. So I will request a clearer and less ambiguous name. I suggest "makeRepoFromImages". It is rather long, but quite clear and unambiguous. If the operation is sufficiently rare then the long name will not hurt, and auto-completion will work once one types "makeRepo". "repoFromImages" might do, but I miss the verb. I do not like "imagesToRepo" because it sounds as if it might be adding images to an existing repo.
            price Paul Price added a comment -

            I would like to avoid the word "ingest", except that's what it does. The IngestTask has multiple functions: parsing an image, putting it in the data repo, and registering it in the registry. Registering is only a part of what it does, so it doesn't seem appropriate to call the whole "registerImages". I was therefore thinking to qualify the kind of ingestion.

            price Paul Price added a comment - I would like to avoid the word "ingest", except that's what it does. The IngestTask has multiple functions: parsing an image, putting it in the data repo, and registering it in the registry. Registering is only a part of what it does, so it doesn't seem appropriate to call the whole "registerImages". I was therefore thinking to qualify the kind of ingestion.
            rowen Russell Owen added a comment -

            I saw Robert's suggestion after I posted mine. I agree that "registerImages" is better than "ingestImages" but I worry that it could easily be mistaken for warping.

            rowen Russell Owen added a comment - I saw Robert's suggestion after I posted mine. I agree that "registerImages" is better than "ingestImages" but I worry that it could easily be mistaken for warping.
            jbosch Jim Bosch added a comment -

            How about ingestToRegistry.py and RegistryIngestTask?

            jbosch Jim Bosch added a comment - How about ingestToRegistry.py and RegistryIngestTask ?

            If the bin script creates a new registry then I strongly feel the name should reflect that. ingestToRegistry implies adding images to an existing registry.

            If the binary code is more flexible, then name it accordingly. It sounds to me as if potentially 3 things could happen in the library code:

            • Create a new repository (which if done in place presumably just means adding a database and a bit of metadata to an existing dir)
            • Copy images or create symlinks to images. If creating a new repo, presumably that is done "in place" and so this step is not done, but it would be wanted if adding images to an existing repo.
            • Ingest data about the images to the repo db
            rowen Russell Owen added a comment - If the bin script creates a new registry then I strongly feel the name should reflect that. ingestToRegistry implies adding images to an existing registry. If the binary code is more flexible, then name it accordingly. It sounds to me as if potentially 3 things could happen in the library code: Create a new repository (which if done in place presumably just means adding a database and a bit of metadata to an existing dir) Copy images or create symlinks to images. If creating a new repo, presumably that is done "in place" and so this step is not done, but it would be wanted if adding images to an existing repo. Ingest data about the images to the repo db
            price Paul Price added a comment -

            Thank you all for your input.

            The script creates a new registry if it is not present, but it does more. The registry really is an implementation detail of what I've been calling "data repositories", and what the script does is ingest images into the data repository. I therefore propose ingestToDataRepository.py. I think this captures the intent of all your suggestions.

            I will move the end date slightly to allow for objections to this solid proposal.

            price Paul Price added a comment - Thank you all for your input. The script creates a new registry if it is not present, but it does more. The registry really is an implementation detail of what I've been calling "data repositories", and what the script does is ingest images into the data repository. I therefore propose ingestToDataRepository.py . I think this captures the intent of all your suggestions. I will move the end date slightly to allow for objections to this solid proposal.
            price Paul Price added a comment -

            Sorry, forgot to mention what was being ingested. New proposal: ingestImagesToDataRepository.py. Yeah, it's a bit long, but it's descriptive (and shell command completion is your friend).

            price Paul Price added a comment - Sorry, forgot to mention what was being ingested. New proposal: ingestImagesToDataRepository.py . Yeah, it's a bit long, but it's descriptive (and shell command completion is your friend).
            price Paul Price added a comment -

            It shall be ingestImagesToDataRepository.py.

            price Paul Price added a comment - It shall be ingestImagesToDataRepository.py .
            tjenness Tim Jenness added a comment -

            We decided that with gen3 now existing and with the triggered ticket marked invalid, we will not be implementing this RFC.

            tjenness Tim Jenness added a comment - We decided that with gen3 now existing and with the triggered ticket marked invalid, we will not be implementing this RFC.

            People

              price Paul Price
              price Paul Price
              Hsin-Fang Chiang, Jim Bosch, Paul Price, Robert Lupton, Russell Owen, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                Planned End:

                Jenkins

                  No builds found.