As for whether we should make the parameters here the default, how hard would it be to make a set of these plots for "jointcal old parameters" vs "jointcal new parameters"?

Ask and ye shall receive (an attachment)...

It looks to me like going from a 5->7th order visit polynomial clearly improves the astrometry, while going from a 7->9th order visit polynomial for photometry doesn't result in much improvement. Three tract+filter combinations show significant AF1 regression at 7th order: HSC-I 9693, HSC-Y 9559, HSC-Z 9558

Here's the full list of values where the lower order does "better" than the higher order. As you say though, we don't know truth here, so it may be that a higher outlier fraction is in fact "more correct".

7th order tracts that exceed the 5th order metric for AF1

HSC-Y 16010 1.33 < 1.47

HSC-G 8521 1.32 < 2.09

HSC-Z 9698 0.342 < 1.3

HSC-I 9693 2.45 < 4.97

HSC-I 9694 0.591 < 0.688

HSC-Z 9694 5.28 < 5.38

HSC-I 9695 1.38 < 1.53

HSC-I 9559 0.314 < 0.833

HSC-Y 9559 2.59 < 4.99

HSC-Z 9558 4.64 < 8.12

7th order tracts that exceed the 5th order metric for AF2

HSC-I 9693 0.743 < 0.811

7th order tracts that exceed the 5th order metric for PA1

HSC-R 8525 17.1 < 17.3

HSC-R 8524 16.6 < 16.7

HSC-G 9558 12.4 < 12.5

7th order tracts that exceed the 5th order metric for PF1

HSC-I 16010 42.7 < 43.0

HSC-R 9698 42.2 < 42.3

HSC-G 16009 30.1 < 30.3

HSC-R 16009 32.1 < 32.2

HSC-R 9693 33.2 < 33.4

HSC-R 9373 37.6 < 37.8

HSC-G 9374 32.6 < 32.7

HSC-R 9374 33.7 < 33.8

HSC-Y 9372 20.6 < 20.7

HSC-R 9697 39.8 < 40.1

HSC-I 15832 42.5 < 42.7

HSC-R 9371 39.7 < 40.0

HSC-R 8525 40.0 < 40.4

HSC-I 8523 42.6 < 42.7

HSC-G 8524 36.8 < 37.0

HSC-R 8524 38.3 < 38.7

Now running validate_drp on the higher order jointcal output.