Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-36861

The MT rotator is going in the wrong direction when tracking ramps

    XMLWordPrintable

    Details

      Description

      In the last two days every time we try a ramp tracking sequence for the MT rotator from the MTRotator commander, the rotator moves in the wrong direction, often quite far in the wrong direction. Here is an example, with a very short ramp (so I didn't have to stop it):

      MT Rotator failed ramp 2022-11-03

      ramp -25 -26 1
      run_command running command method <bound method RotatorCommander.do_ramp of <lsst.ts.mtrotator.rotator_commander.RotatorCommander object at 0x7fc1390205e0>>
      Tracking a ramp from -25.0 to -26.0 at speed 1.0; this will take 1.00 seconds
      Finished command ramp
      track angle=-25.00 velocity=-1.00
      1667506455.090: controllerState: controllerState=2, offlineSubstate=1, enabledSubstate=2, applicationStatus=5124
      1667506455.090: tracking: tracking=False, lost=True, noNewCommand=False
      1667506455.091: target: position=-25.0000, velocity=-1.0000, tai=1667506455.0936
      1667506455.090: inPosition: inPosition=False
      track angle=-25.15 velocity=-1.00
      1667506455.240: rotation: demandPosition=-25.0000, demandVelocity=-0.0000, demandAcceleration=-0.0600, actualPosition=-25.0000, actualVelocity=-0.0000, debugActualVelocityA=-0.0000, debugActualVelocityB=-0.0000, odometer=392282.8500, timestamp=1667506455.1500
      1667506455.240: tracking: tracking=True, lost=True, noNewCommand=False
      1667506455.240: inPosition: inPosition=True
      1667506455.290: rotation: demandPosition=-25.0000, demandVelocity=-0.0100, demandAcceleration=-0.1200, actualPosition=-25.0000, actualVelocity=-0.0000, debugActualVelocityA=-0.0000, debugActualVelocityB=-0.0000, odometer=392282.8500, timestamp=1667506455.2000
      1667506455.290: target: position=-25.1530, velocity=-1.0000, tai=1667506455.2466
      1667506455.340: rotation: demandPosition=-25.0000, demandVelocity=-0.0100, demandAcceleration=-0.1200, actualPosition=-25.0000, actualVelocity=-0.0200, debugActualVelocityA=-0.0200, debugActualVelocityB=-0.0100, odometer=392282.8500, timestamp=1667506455.2500
      1667506455.340: ccwFollowingError: positionError=-0.0000, velocityError=-0.0200, timestamp=1667506455.2500
      1667506455.390: rotation: demandPosition=-25.0000, demandVelocity=-0.0200, demandAcceleration=-0.1200, actualPosition=-25.0000, actualVelocity=-0.0300, debugActualVelocityA=-0.0300, debugActualVelocityB=-0.0300, odometer=392282.8600, timestamp=1667506455.3000
      1667506455.390: motors: raw=[-2453700000.0, -2451600000.0], torque=[0.0, 0.0], current=[0.0, 0.0], busVoltage=0.0000
      1667506455.390: ccwFollowingError: positionError=-0.0000, velocityError=-0.0300, timestamp=1667506455.3000
      track angle=-25.35 velocity=-1.00
      1667506455.440: rotation: demandPosition=-25.0000, demandVelocity=-0.0300, demandAcceleration=-0.1200, actualPosition=-25.0000, actualVelocity=-0.0300, debugActualVelocityA=-0.0300, debugActualVelocityB=-0.0300, odometer=392282.8600, timestamp=1667506455.3500
      1667506455.490: motors: raw=[-2453600000.0, -2451500000.0], torque=[0.0, 0.0], current=[0.0, 0.0], busVoltage=0.0000
      1667506455.490: target: position=-25.3514, velocity=-1.0000, tai=1667506455.4450
      1667506455.540: rotation: demandPosition=-25.0100, demandVelocity=-0.0400, demandAcceleration=-0.1200, actualPosition=-25.0100, actualVelocity=-0.0400, debugActualVelocityA=-0.0400, debugActualVelocityB=-0.0400, odometer=392282.8600, timestamp=1667506455.4500
      1667506455.540: motors: raw=[-2453600000.0, -2451400000.0], torque=[0.0, 0.0], current=[0.0, 0.0], busVoltage=0.0000
      1667506455.540: ccwFollowingError: positionError=-0.0100, velocityError=-0.0400, timestamp=1667506455.4500
      1667506455.590: motors: raw=[-2453500000.0, -2451400000.0], torque=[0.0, 0.0], current=[0.0, 0.0], busVoltage=0.0000
      track angle=-25.55 velocity=-1.00
      1667506455.640: rotation: demandPosition=-25.0100, demandVelocity=-0.0500, demandAcceleration=-0.1200, actualPosition=-25.0100, actualVelocity=-0.0500, debugActualVelocityA=-0.0500, debugActualVelocityB=-0.0500, odometer=392282.8600, timestamp=1667506455.5500
      1667506455.640: motors: raw=[-2453400000.0, -2451300000.0], torque=[0.0, 0.0], current=[0.0, 0.0], busVoltage=0.0000
      1667506455.640: ccwFollowingError: positionError=-0.0100, velocityError=-0.0500, timestamp=1667506455.5500
      1667506455.690: rotation: demandPosition=-25.0100, demandVelocity=-0.0600, demandAcceleration=-0.1200, actualPosition=-25.0100, actualVelocity=-0.0600, debugActualVelocityA=-0.0600, debugActualVelocityB=-0.0600, odometer=392282.8700, timestamp=1667506455.6000
      1667506455.690: motors: raw=[-2453300000.0, -2451200000.0], torque=[0.0, 0.0], current=[0.0, 0.0], busVoltage=0.0000
      1667506455.690: ccwFollowingError: positionError=-0.0100, velocityError=-0.0600, timestamp=1667506455.6000
      1667506455.690: target: position=-25.5512, velocity=-1.0000, tai=1667506455.6448
      1667506455.740: rotation: demandPosition=-25.0200, demandVelocity=-0.0600, demandAcceleration=-0.1200, actualPosition=-25.0200, actualVelocity=-0.0600, debugActualVelocityA=-0.0600, debugActualVelocityB=-0.0600, odometer=392282.8700, timestamp=1667506455.6500
      1667506455.740: motors: raw=[-2453200000.0, -2451100000.0], torque=[0.0, 0.0], current=[0.0, 0.0], busVoltage=0.0000
      1667506455.741: ccwFollowingError: positionError=-0.0200, velocityError=-0.0600, timestamp=1667506455.6500
      1667506455.790: rotation: demandPosition=-25.0200, demandVelocity=-0.0700, demandAcceleration=-0.1200, actualPosition=-25.0200, actualVelocity=-0.0700, debugActualVelocityA=-0.0700, debugActualVelocityB=-0.0700, odometer=392282.8700, timestamp=1667506455.7000
      1667506455.790: motors: raw=[-2453100000.0, -2451000000.0], torque=[0.0, 0.0], current=[0.0, 0.0], busVoltage=0.0000
      1667506455.791: ccwFollowingError: positionError=-0.0200, velocityError=-0.0700, timestamp=1667506455.7000
      track angle=-25.75 velocity=-1.00
      1667506455.840: rotation: demandPosition=-25.0200, demandVelocity=-0.0800, demandAcceleration=-0.1200, actualPosition=-25.0200, actualVelocity=-0.0800, debugActualVelocityA=-0.0800, debugActualVelocityB=-0.0800, odometer=392282.8800, timestamp=1667506455.7500
      1667506455.840: motors: raw=[-2452900000.0, -2450800000.0], torque=[0.0, 0.0], current=[0.0, 0.0], busVoltage=0.0000
      1667506455.840: ccwFollowingError: positionError=-0.0200, velocityError=-0.0800, timestamp=1667506455.7500
      1667506455.890: rotation: demandPosition=-25.0300, demandVelocity=-0.0800, demandAcceleration=-0.1200, actualPosition=-25.0300, actualVelocity=-0.0800, debugActualVelocityA=-0.0800, debugActualVelocityB=-0.0800, odometer=392282.8800, timestamp=1667506455.8000
      1667506455.890: motors: raw=[-2452800000.0, -2450700000.0], torque=[0.0, 0.0], current=[0.0, 0.0], busVoltage=0.0000
      1667506455.890: ccwFollowingError: positionError=-0.0300, velocityError=-0.0800, timestamp=1667506455.8000
      1667506455.890: target: position=-25.7517, velocity=-1.0000, tai=1667506455.8453
      1667506455.940: rotation: demandPosition=-25.0300, demandVelocity=-0.0900, demandAcceleration=-0.1200, actualPosition=-25.0300, actualVelocity=-0.0900, debugActualVelocityA=-0.0900, debugActualVelocityB=-0.0800, odometer=392282.8800, timestamp=1667506455.8500
      1667506455.940: motors: raw=[-2452600000.0, -2450500000.0], torque=[0.0, 0.0], current=[0.0, 0.0], busVoltage=0.0000
      1667506455.940: ccwFollowingError: positionError=-0.0300, velocityError=-0.0900, timestamp=1667506455.8500
      1667506455.990: rotation: demandPosition=-25.0400, demandVelocity=-0.0900, demandAcceleration=-0.1200, actualPosition=-25.0400, actualVelocity=-0.0900, debugActualVelocityA=-0.0900, debugActualVelocityB=-0.0900, odometer=392282.8900, timestamp=1667506455.9000
      1667506455.990: motors: raw=[-2452500000.0, -2450300000.0], torque=[0.0, 0.0], current=[0.0, 0.0], busVoltage=0.0000
      1667506455.990: ccwFollowingError: positionError=-0.0400, velocityError=-0.0900, timestamp=1667506455.9000
      track angle=-25.95 velocity=-1.00
      1667506456.040: rotation: demandPosition=-25.0400, demandVelocity=-0.1000, demandAcceleration=-0.1100, actualPosition=-25.0400, actualVelocity=-0.1000, debugActualVelocityA=-0.1000, debugActualVelocityB=-0.1000, odometer=392282.8900, timestamp=1667506455.9500
      1667506456.040: motors: raw=[-2452300000.0, -2450200000.0], torque=[0.0, 0.0], current=[0.0, 0.0], busVoltage=0.0000
      1667506456.041: ccwFollowingError: positionError=-0.0400, velocityError=-0.1000, timestamp=1667506455.9500
      1667506456.090: rotation: demandPosition=-25.0500, demandVelocity=-0.1000, demandAcceleration=-0.0500, actualPosition=-25.0500, actualVelocity=-0.1000, debugActualVelocityA=-0.1000, debugActualVelocityB=-0.1100, odometer=392282.9000, timestamp=1667506456.0000
      1667506456.090: motors: raw=[-2452100000.0, -2450000000.0], torque=[0.0, 0.0], current=[0.0, 0.0], busVoltage=0.0000
      1667506456.090: ccwFollowingError: positionError=-0.0500, velocityError=-0.1100, timestamp=1667506456.0000
      1667506456.091: target: position=-25.9516, velocity=-1.0000, tai=1667506456.0452
      1667506456.140: rotation: demandPosition=-25.0500, demandVelocity=-0.1000, demandAcceleration=0.0200, actualPosition=-25.0500, actualVelocity=-0.1000, debugActualVelocityA=-0.1100, debugActualVelocityB=-0.1000, odometer=392282.9000, timestamp=1667506456.0500
      1667506456.140: motors: raw=[-2451900000.0, -2449800000.0], torque=[0.0, 0.0], current=[0.0, 0.0], busVoltage=0.0000
      1667506456.140: ccwFollowingError: positionError=-0.0500, velocityError=-0.1000, timestamp=1667506456.0500
      1667506456.190: rotation: demandPosition=-25.0600, demandVelocity=-0.1000, demandAcceleration=0.1600, actualPosition=-25.0600, actualVelocity=-0.1000, debugActualVelocityA=-0.1000, debugActualVelocityB=-0.1000, odometer=392282.9100, timestamp=1667506456.1000
      1667506456.190: motors: raw=[-2451700000.0, -2449600000.0], torque=[0.0, 0.0], current=[0.0, 0.0], busVoltage=0.0000
      track angle=-26.00 velocity=0.00
      1667506456.240: rotation: demandPosition=-25.0600, demandVelocity=-0.0900, demandAcceleration=0.3000, actualPosition=-25.0600, actualVelocity=-0.0900, debugActualVelocityA=-0.0900, debugActualVelocityB=-0.1000, odometer=392282.9100, timestamp=1667506456.1500
      1667506456.240: motors: raw=[-2451500000.0, -2449400000.0], torque=[0.0, 0.0], current=[0.0, 0.0], busVoltage=0.0000
      1667506456.240: ccwFollowingError: positionError=-0.0600, velocityError=-0.0900, timestamp=1667506456.1500
      1667506456.290: rotation: demandPosition=-25.0600, demandVelocity=-0.0700, demandAcceleration=0.3700, actualPosition=-25.0600, actualVelocity=-0.0700, debugActualVelocityA=-0.0700, debugActualVelocityB=-0.0700, odometer=392282.9200, timestamp=1667506456.2000
      1667506456.290: motors: raw=[-2451400000.0, -2449200000.0], torque=[0.0, 0.0], current=[0.0, 0.0], busVoltage=0.0000
      1667506456.291: ccwFollowingError: positionError=-0.0600, velocityError=-0.0700, timestamp=1667506456.2000
      1667506456.290: tracking: tracking=False, lost=True, noNewCommand=False
      1667506456.291: target: position=-26.0000, velocity=0.0000, tai=1667506456.2452
      1667506456.290: inPosition: inPosition=False
      

      This shows an attempt to track a path from -25 to -26 degrees at 1 deg/second (thus a very short ramp). The rotator starts moving in the correct direction but soon turns around. The tracking commands look like this: track angle=-25.95 velocity=-1.00 (though you can also get the same information from the "target: " events, which the CSC publishes when it sends a track command or point-to-point move command to the low-level controller).

      I also attached a graph from Julen Garcia showing position vs time (of both rotator and camera cable wrap). This shows the rotator starting off in the correct direction, but then turning around and ending up in the wrong direction from the starting point. Also notice that the end of the motion is very abrupt; I doubt it is properly jerk-limited.

        Attachments

        1. failed ramp.txt
          235 kB
        2. Rotator PosVelAcc.jpg
          Rotator PosVelAcc.jpg
          193 kB
        3. rotator ramp path.png
          rotator ramp path.png
          261 kB

          Issue Links

            Activity

              People

              Assignee:
              ttsai Te-Wei Tsai
              Reporter:
              rowen Russell Owen
              Reviewers:
              Russell Owen
              Watchers:
              Brian Stalder, Bruno Quint, Julen Garcia, Russell Owen, Te-Wei Tsai
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.