Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-27071

butler ingest-raws crashes on one bad file

    XMLWordPrintable

    Details

    • Story Points:
      2
    • Team:
      Architecture
    • Urgent?:
      No

      Description

      In IHS-4230 it was noted that if a file fails metadata extraction the entire ingest fails.

      We need to have it ingest everything it can and then report the problematic files.

        Attachments

          Activity

          Ascending order - Click to sort in descending order
          No builds found.
          tjenness Tim Jenness created issue -
          tjenness Tim Jenness made changes -
          Field Original Value New Value
          Link This issue is triggered by IHS-4230 [ IHS-4230 ]
          tjenness Tim Jenness made changes -
          Status To Do [ 10001 ] In Progress [ 3 ]
          tjenness Tim Jenness made changes -
          Watchers Tim Jenness [ Tim Jenness ] Eric Morganson, Tim Jenness [ Eric Morganson, Tim Jenness ]
          Hide
          Permalink
          tjenness Tim Jenness added a comment -

          Jim Bosch can you take a look at this? I've added logging everywhere and now let ingest do what it can and it is only right at the end that it reports all the problems and exists with bad status.

          Show
          tjenness Tim Jenness added a comment - Jim Bosch can you take a look at this? I've added logging everywhere and now let ingest do what it can and it is only right at the end that it reports all the problems and exists with bad status.
          tjenness Tim Jenness made changes -
          Reviewers Jim Bosch [ jbosch ]
          Status In Progress [ 3 ] In Review [ 10004 ]
          Hide
          Permalink
          jbosch Jim Bosch added a comment -

          Minor comments on the PR.

          Show
          jbosch Jim Bosch added a comment - Minor comments on the PR.
          jbosch Jim Bosch made changes -
          Status In Review [ 10004 ] Reviewed [ 10101 ]
          tjenness Tim Jenness made changes -
          Resolution Done [ 10000 ]
          Status Reviewed [ 10101 ] Done [ 10002 ]

            People

            Assignee:
            tjenness Tim Jenness
            Reporter:
            tjenness Tim Jenness
            Reviewers:
            Jim Bosch
            Watchers:
            Eric Morganson [X] (Inactive), Jim Bosch, Tim Jenness
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.