Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-4127

The image XYflip bug in dealing with FITS in integer values

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Firefly
    • Labels:
    • Story Points:
      10
    • Sprint:
      SUIT Sprint 2016-9
    • Team:
      Science User Interface

      Description

      The bug appeared first when flipping an image with mask overlaid on it. Further study showed that the flip function has a bug when the FITS image values are integers.

        Attachments

          Activity

          Hide
          zhang Lijun Zhang [X] (Inactive) added a comment -

          After investigating this issue, I found that the stateAry in VisServerOps contains identical information of the first image. The stateAry[1] is supposed to contain the information of the second layer, mask plot. But it has exactly the same information as stateAry[0].

           for(int i= 0; (i<stateAry.length); i++)  {
                      resultAry[i]= flipImageOnY(stateAry[i]);
                      if (success) success= resultAry[i].isSuccess();
           }
          

          Show
          zhang Lijun Zhang [X] (Inactive) added a comment - After investigating this issue, I found that the stateAry in VisServerOps contains identical information of the first image. The stateAry[1] is supposed to contain the information of the second layer, mask plot. But it has exactly the same information as stateAry[0]. for(int i= 0; (i<stateAry.length); i++) { resultAry[i]= flipImageOnY(stateAry[i]); if (success) success= resultAry[i].isSuccess(); }
          Hide
          roby Trey Roby added a comment -

          Lijun - I think this is the flip process with integers on the server side. If you forget about masking and read only the mask hdu (hdu 1) as a regular image and then flip, you see the same issues. I would like you to look at it and see if you can track it down.

          Trey

          Show
          roby Trey Roby added a comment - Lijun - I think this is the flip process with integers on the server side. If you forget about masking and read only the mask hdu (hdu 1) as a regular image and then flip, you see the same issues. I would like you to look at it and see if you can track it down. Trey
          Hide
          zhang Lijun Zhang [X] (Inactive) added a comment - - edited

          After extensive investigation, the bug is the data type issue. All calculation is used float. But the original data type in the HDU should be observed according to the BitPix value. When the FitsRead is flipped, the new FitsRead is created using the flipped data without converting to its original data type. Thus, header and the data are not in consistence. The same bug is appeared in writeSimpleFits and writeFits in FitsRead.java. In short there are three bugs found:

          1. at doFlip() method in FlipXY at the line

            Object inData = ArrayFuncs.convertArray(hdu.getData().getData(), Float.TYPE);
             Object newData=null;
             if (direction.equalsIgnoreCase("yAxis")) {
                        newData = doFlipInY(inData, dim);
             } else if (direction.equalsIgnoreCase("xAxis")) {
                        newData = doFlipInX(inData, dim);
             } else {
                        throw new FitsException(
                                "Cannot flip a PLATE projection image");
             }
            ImageData newImageData =  new ImageData(newData);
            

          2. writeSimpleFits in FitsRaad.java
          3. WriteFits in FitsRead.java
            The original datatype for mask is short. The FitsRead converted it to float (inData) and using it to do all the calculations. After the float data array is flipped, the newData is still an float array. When it is used to create the output FitsRead, the data type in the ImageHDU is changed to float instead of short.
            The correction in doFlip() is (the correction is similar in other two places):

            Object data =  ArrayFuncs.convertArray(fdata, getDataType(hdu.getBitPix()), true);
            ImageData newImageData =  new ImageData(data);
            

          Show
          zhang Lijun Zhang [X] (Inactive) added a comment - - edited After extensive investigation, the bug is the data type issue. All calculation is used float. But the original data type in the HDU should be observed according to the BitPix value. When the FitsRead is flipped, the new FitsRead is created using the flipped data without converting to its original data type. Thus, header and the data are not in consistence. The same bug is appeared in writeSimpleFits and writeFits in FitsRead.java. In short there are three bugs found: at doFlip() method in FlipXY at the line Object inData = ArrayFuncs.convertArray(hdu.getData().getData(), Float.TYPE); Object newData=null; if (direction.equalsIgnoreCase("yAxis")) { newData = doFlipInY(inData, dim); } else if (direction.equalsIgnoreCase("xAxis")) { newData = doFlipInX(inData, dim); } else { throw new FitsException( "Cannot flip a PLATE projection image"); } ImageData newImageData = new ImageData(newData); writeSimpleFits in FitsRaad.java WriteFits in FitsRead.java The original datatype for mask is short. The FitsRead converted it to float (inData) and using it to do all the calculations. After the float data array is flipped, the newData is still an float array. When it is used to create the output FitsRead, the data type in the ImageHDU is changed to float instead of short. The correction in doFlip() is (the correction is similar in other two places): Object data = ArrayFuncs.convertArray(fdata, getDataType(hdu.getBitPix()), true); ImageData newImageData = new ImageData(data);
          Hide
          zhang Lijun Zhang [X] (Inactive) added a comment -

          Trey,

          Can you please review this ticket? Thanks!

          Show
          zhang Lijun Zhang [X] (Inactive) added a comment - Trey, Can you please review this ticket? Thanks!
          Hide
          zhang Lijun Zhang [X] (Inactive) added a comment -

          Merged to dev.

          Show
          zhang Lijun Zhang [X] (Inactive) added a comment - Merged to dev.

            People

            • Assignee:
              zhang Lijun Zhang [X] (Inactive)
              Reporter:
              zhang Lijun Zhang [X] (Inactive)
              Reviewers:
              Trey Roby
              Watchers:
              Lijun Zhang [X] (Inactive), Trey Roby, Xiuqin Wu [X] (Inactive)
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel