Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-20139

Add presigned POST-based S3 build uploads to LTD Conveyor

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      In DM-20122 we determined that presigned POST URLs are a good fit for giving upload clients the ability to upload a build without having to maintain their own AWS/S3 credentials. This ticket adds that capability to the LTD Conveyor application. Other tickets will add the ability for LTD Keeper to generate the presigned URLs.

        Attachments

          Issue Links

          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-20140 Add build registration route that provides presigned URLs

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-20122 Investigate client auth patterns for LSST the Docs

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            jsick Jonathan Sick added a comment -

            Relevant AWS docs on using presigned POST URLs: https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-UsingHTTPPOST.html

            Show
            jsick Jonathan Sick added a comment - Relevant AWS docs on using presigned POST URLs: https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-UsingHTTPPOST.html
            Hide
            Permalink
            jsick Jonathan Sick added a comment -

            From the [S3 docs|HTML Form Fields]:

             

             

            The variable ${filename} is automatically replaced with the name of the file provided by the user and is recognized by all form fields. If the browser or client provides a full or partial path to the file, only the text following the last slash or backslash () is used (for example, C:\Program Files\directory1\file.txt is interpreted as file.txt). If no file or file name is provided, the variable is replaced with an empty string.

             

            This means that there needs to be a different presigned URL for each "directory"

            Show
            jsick Jonathan Sick added a comment - From the [S3 docs| HTML Form Fields ]:     The variable ${filename } is automatically replaced with the name of the file provided by the user and is recognized by all form fields. If the browser or client provides a full or partial path to the file, only the text following the last slash or backslash () is used (for example, C:\Program Files\directory1\file.txt is interpreted as file.txt ). If no file or file name is provided, the variable is replaced with an empty string.   This means that there needs to be a different presigned URL for each "directory"
            Hide
            Permalink
            jsick Jonathan Sick added a comment -

            Best docs on setting up the client-side post: https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-HTTPPOSTForms.html

            Show
            jsick Jonathan Sick added a comment - Best docs on setting up the client-side post: https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-HTTPPOSTForms.html
            Hide
            Permalink
            jsick Jonathan Sick added a comment -

            Released as 0.5.0a1.

            Show
            jsick Jonathan Sick added a comment - Released as 0.5.0a1.

              People

              Assignee:
              jsick Jonathan Sick
              Reporter:
              jsick Jonathan Sick
              Watchers:
              Jonathan Sick
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.