Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-21528

Escape quotes in Jinja variables being inserted into technote, and other, templates

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: templates
    • Labels:
      None

      Description

      The sqrbot setup PR for ittn-004 failed with a spinx-build error:

      https://travis-ci.com/lsst-it/ittn-004/builds/130548516
       
      $ source ~/virtualenv/python3.7/bin/activate
       
      $ python --version
       
      Python 3.7.1
       
      $ pip --version
       
      pip 19.0.3 from /home/[secure]/virtualenv/python3.7.1/lib/python3.7/site-packages/pip (python 3.7)
      install.1
       
      15.98s$ pip install -r requirements.txt
      install.2
       
      2.80s$ pip install "ltd-conveyor>=0.4.2,<0.5"
       
      0.48s$ sphinx-build -b html -a -n -d _build/doctree . _build/html
       
      Running Sphinx v1.7.9
       
      /home/[secure]/virtualenv/python3.7.1/lib/python3.7/site-packages/documenteer/sphinxconfig/technoteconf.py:65: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
       
        _metadata = yaml.load(meta_stream)
       
      Configuration error:
       
      There is a programable error in your configuration file:
       
      Traceback (most recent call last):
       
        File "/home/[secure]/virtualenv/python3.7.1/lib/python3.7/site-packages/sphinx/config.py", line 161, in __init__
       
          execfile_(filename, config)
       
        File "/home/[secure]/virtualenv/python3.7.1/lib/python3.7/site-packages/sphinx/util/pycompat.py", line 150, in execfile_
       
          exec_(code, _globals)
       
        File "conf.py", line 14, in <module>
       
          confs = configure_technote(f)
       
        File "/home/[secure]/virtualenv/python3.7.1/lib/python3.7/site-packages/documenteer/sphinxconfig/technoteconf.py", line 65, in configure_technote
       
          _metadata = yaml.load(meta_stream)
       
        File "/home/[secure]/virtualenv/python3.7.1/lib/python3.7/site-packages/yaml/__init__.py", line 114, in load
       
          return loader.get_single_data()
       
        File "/home/[secure]/virtualenv/python3.7.1/lib/python3.7/site-packages/yaml/constructor.py", line 41, in get_single_data
       
          node = self.get_single_node()
       
        File "/home/[secure]/virtualenv/python3.7.1/lib/python3.7/site-packages/yaml/composer.py", line 36, in get_single_node
       
          document = self.compose_document()
       
        File "/home/[secure]/virtualenv/python3.7.1/lib/python3.7/site-packages/yaml/composer.py", line 55, in compose_document
       
          node = self.compose_node(None, None)
       
        File "/home/[secure]/virtualenv/python3.7.1/lib/python3.7/site-packages/yaml/composer.py", line 84, in compose_node
       
          node = self.compose_mapping_node(anchor)
       
        File "/home/[secure]/virtualenv/python3.7.1/lib/python3.7/site-packages/yaml/composer.py", line 127, in compose_mapping_node
       
          while not self.check_event(MappingEndEvent):
       
        File "/home/[secure]/virtualenv/python3.7.1/lib/python3.7/site-packages/yaml/parser.py", line 98, in check_event
       
          self.current_event = self.state()
       
        File "/home/[secure]/virtualenv/python3.7.1/lib/python3.7/site-packages/yaml/parser.py", line 439, in parse_block_mapping_key
       
          "expected <block end>, but found %r" % token.id, token.start_mark)
       
      yaml.parser.ParserError: while parsing a block mapping
       
        in "/home/[secure]/build/lsst-it/ittn-004/metadata.yaml", line 6, column 1
       
      expected <block end>, but found '<scalar>'
       
        in "/home/[secure]/build/lsst-it/ittn-004/metadata.yaml", line 40, column 48
       
      The command "sphinx-build -b html -a -n -d _build/doctree . _build/html" exited with 2.
       
      Done. Your build exited with 1.
      

        Attachments

          Activity

          Hide
          jhoblitt Joshua Hoblitt added a comment -

          Jonathan Sick The problem was that the creation wizard allowed ' to be input as part of the technote description. Possible preventive measure might be filtering out the quotes, escaping them, or some sort of input validation error returned to the user.

          $ git diff 9c747c09a9d40e006096dda2cabdaca4dff036bf^!
          diff --git a/metadata.yaml b/metadata.yaml
          index 9e613ee..0a5ffee 100644
          --- a/metadata.yaml
          +++ b/metadata.yaml
          @@ -13,7 +13,7 @@ serial_number: '004'
           doc_id: 'ITTN-004'
           
           # Title of the document, without the series/serial designation
          -doc_title: '[Proposed] LSST On-Prem Domain Name Service (DNS) '
          +doc_title: '[Proposed] LSST On-Prem Domain Name Service (DNS)'
           
           # Author names, ordered as a list. Each author name should be formatted as 'First Last'
           authors:
          @@ -37,7 +37,7 @@ authors:
           copyright: '2019, Association of Universities for Research in Astronomy'
           
           # Description. A short, 1-2 sentence statemement used by document indices.
          -description: 'Simplified DNS strategy for LSST's On-Prem datacenter operations.'
          +description: 'Simplified DNS strategy for LSST On-Prem datacenter operations.'
           
           # Abstract, if available
           # abstract: >

          Show
          jhoblitt Joshua Hoblitt added a comment - Jonathan Sick The problem was that the creation wizard allowed ' to be input as part of the technote description. Possible preventive measure might be filtering out the quotes, escaping them, or some sort of input validation error returned to the user. $ git diff 9c747c09a9d40e006096dda2cabdaca4dff036bf^! diff --git a/metadata.yaml b/metadata.yaml index 9e613ee..0a5ffee 100644 --- a/metadata.yaml +++ b/metadata.yaml @@ - 13 , 7 + 13 , 7 @@ serial_number: '004' doc_id: 'ITTN-004' # Title of the document, without the series/serial designation -doc_title: '[Proposed] LSST On-Prem Domain Name Service (DNS) ' +doc_title: '[Proposed] LSST On-Prem Domain Name Service (DNS)' # Author names, ordered as a list. Each author name should be formatted as 'First Last' authors: @@ - 37 , 7 + 37 , 7 @@ authors: copyright: '2019, Association of Universities for Research in Astronomy' # Description. A short , 1 - 2 sentence statemement used by document indices. -description: 'Simplified DNS strategy for LSST' s On-Prem datacenter operations.' +description: 'Simplified DNS strategy for LSST On-Prem datacenter operations.' # Abstract, if available # abstract : >
          Hide
          jsick Jonathan Sick added a comment -

          Ok, will do.

          Show
          jsick Jonathan Sick added a comment - Ok, will do.
          Hide
          jsick Jonathan Sick added a comment -
          Show
          jsick Jonathan Sick added a comment - Released version 0.3.0 of templatekit with a new Jinja filter for escaping YAML strings https://templatekit.lsst.io/v/0.3.0/template-guide/templatekit-jinja-extensions.html#escape-yaml-doublequoted and published tips on templating YAML files: https://templatekit.lsst.io/v/0.3.0/template-guide/templating-yaml-files.html Updated the technote_rst and nbreport templates to use this new Jinja filter: https://github.com/lsst/templates/pull/50
          Hide
          jsick Jonathan Sick added a comment -

          Updated templatebot to use version 0.3.0 of templatekit.

          Show
          jsick Jonathan Sick added a comment - Updated templatebot to use version 0.3.0 of templatekit.

            People

            • Assignee:
              jsick Jonathan Sick
              Reporter:
              jhoblitt Joshua Hoblitt
              Watchers:
              Frossie Economou, Jonathan Sick, Joshua Hoblitt, Luis Corral
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel