Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-27345

Fix schema incompatibility in the NTS EFD

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      We were getting write errors in InfluxDB since SAL XML 6.2 due schema incompatibilty.

      [2020-10-27 22:52:07,962] ERROR Encountered error partial write: field type conflict: input field "lst" on measurement "lsst.sal.MTPtg.timeAndDate" is type float, already exists as type string dropped=1 (com.datamountaineer.streamreactor.connect.influx.writers.InfluxDbWriter)
      		 
      org.influxdb.InfluxDBException$FieldTypeConflictException: partial write: field type conflict: input field "lst" on measurement "lsst.sal.MTPtg.timeAndDate" is type float, already exists as type string dropped=1

        Attachments

          Activity

          Hide
          Permalink
          afausti Angelo Fausti added a comment - - edited

          One solution is to rename the fields in InfluxDB, for that I made a copy of the measurements with the old data, and dropped the current measurements to get rid the errors above.

          > SELECT * INTO "lsst.sal.ATPtg.timeAndDate.old" FROM "lsst.sal.ATPtg.timeAndDate" WHERE time > now()-30d
          		 
          name: result
          		 
          time written
          		 
          ---- -------
          		 
          0    8495062
          		 
          DROP MEASUREMENT "lsst.sal.ATPtg.timeAndDate" 
           
          		 
           
          		 
          > SELECT * INTO "lsst.sal.MTPtg.timeAndDate.old" FROM "lsst.sal.MTPtg.timeAndDate" WHERE time > now()-30d
          		 
          name: result
          		 
          time written
          		 
          ---- -------
          		 
          0    7039696
          		 
          DROP MEASUREMENT "lsst.sal.MTPtg.timeAndDate"
          		 
           
          		 
           
          		 
          > SELECT * INTO "lsst.sal.ATPtg.currentTargetStatus.old" FROM "lsst.sal.ATPtg.currentTargetStatus" WHERE time > now()-30d
          		 
          name: result
          		 
          time written
          		 
          ---- -------
          		 
          0    7069353
          		 
          DROP MEASUREMENT "lsst.sal.ATPtg.currentTargetStatus"
          		 
           
          		 
           
          		 
          > SELECT * INTO "lsst.sal.MTPtg.currentTargetStatus.old" FROM "lsst.sal.MTPtg.currentTargetStatus" WHERE time > now()-30d
          		 
          name: result
          		 
          time written
          		 
          ---- -------
          		 
          0    5622937
          		 
          DROP MEASUREMENT "lsst.sal.MTPtg.currentTargetStatus"
          		 
           
          		 
           

          Show
          afausti Angelo Fausti added a comment - - edited One solution is to rename the fields in InfluxDB, for that I made a copy of the measurements with the old data, and dropped the current measurements to get rid the errors above. > SELECT * INTO "lsst.sal.ATPtg.timeAndDate.old" FROM "lsst.sal.ATPtg.timeAndDate" WHERE time > now()-30d name: result time written ---- ------- 0 8495062 DROP MEASUREMENT "lsst.sal.ATPtg.timeAndDate"     > SELECT * INTO "lsst.sal.MTPtg.timeAndDate.old" FROM "lsst.sal.MTPtg.timeAndDate" WHERE time > now()-30d name: result time written ---- ------- 0 7039696 DROP MEASUREMENT "lsst.sal.MTPtg.timeAndDate"     > SELECT * INTO "lsst.sal.ATPtg.currentTargetStatus.old" FROM "lsst.sal.ATPtg.currentTargetStatus" WHERE time > now()-30d name: result time written ---- ------- 0 7069353 DROP MEASUREMENT "lsst.sal.ATPtg.currentTargetStatus"     > SELECT * INTO "lsst.sal.MTPtg.currentTargetStatus.old" FROM "lsst.sal.MTPtg.currentTargetStatus" WHERE time > now()-30d name: result time written ---- ------- 0 5622937 DROP MEASUREMENT "lsst.sal.MTPtg.currentTargetStatus"    

            People

            Assignee:
            afausti Angelo Fausti
            Reporter:
            afausti Angelo Fausti
            Watchers:
            Angelo Fausti
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.