estimateAzimuthMisreg.py: list' object has no attribute 'size'. run_07_pairs_misreg_1_

[disilvestro]

ERROR

At out_run_07_pairs_misreg.e, for every pair(?) listed below

Error: "Traceback" found in /scratch/09580/gdisilvestro/PacayaSenD26/run_files/run_07_pairs_misreg_0_20141031_20141124_1.e
Error: "Traceback" found in /scratch/09580/gdisilvestro/PacayaSenD26/run_files/run_07_pairs_misreg_0_20141031_20141218_2.e
Error: "Traceback" found in /scratch/09580/gdisilvestro/PacayaSenD26/run_files/run_07_pairs_misreg_0_20141031_20150429_3.e
...
Error: "Traceback" found in /scratch/09580/gdisilvestro/PacayaSenD26/run_files/run_07_pairs_misreg_1_20260413_20260419_598.e
Error: "Traceback" found in /scratch/09580/gdisilvestro/PacayaSenD26/run_files/run_07_pairs_misreg_1_20260413_20260425_599.e
Error: "Traceback" found in /scratch/09580/gdisilvestro/PacayaSenD26/run_files/run_07_pairs_misreg_1_20260413_20260501_600.e
Error: "Traceback" found in /scratch/09580/gdisilvestro/PacayaSenD26/run_files/run_07_pairs_misreg_1_20260419_20260425_601.e
Error: "Traceback" found in /scratch/09580/gdisilvestro/PacayaSenD26/run_files/run_07_pairs_misreg_1_20260419_20260501_602.e
Error: "Traceback" found in /scratch/09580/gdisilvestro/PacayaSenD26/run_files/run_07_pairs_misreg_1_20260419_20260507_603.e
Error: "Traceback" found in /scratch/09580/gdisilvestro/PacayaSenD26/run_files/run_07_pairs_misreg_1_20260425_20260501_604.e
Error: "Traceback" found in /scratch/09580/gdisilvestro/PacayaSenD26/run_files/run_07_pairs_misreg_1_20260425_20260507_605.e
Error: "Traceback" found in /scratch/09580/gdisilvestro/PacayaSenD26/run_files/run_07_pairs_misreg_1_20260501_20260507_606.e

I get the following error

Traceback (most recent call last):
  File "/work2/09580/gdisilvestro/stampede3/code/minsar/tools/miniforge3/envs/minsar/share/isce2/topsStack/SentinelWrapper.py", line 251, in <module>
    main(args.start,args.end)
  File "/work2/09580/gdisilvestro/stampede3/code/minsar/tools/miniforge3/envs/minsar/share/isce2/topsStack/SentinelWrapper.py", line 242, in main
    cfgParser.runCmd()
  File "/work2/09580/gdisilvestro/stampede3/code/minsar/tools/miniforge3/envs/minsar/share/isce2/topsStack/SentinelWrapper.py", line 54, in runCmd
    func_modules.main(self.funcParams[section])
  File "/work2/09580/gdisilvestro/stampede3/code/minsar/tools/miniforge3/envs/minsar/share/isce2/topsStack/estimateAzimuthMisreg.py", line 137, in main
    if val.size == 0 :
AttributeError: 'list' object has no attribute 'size'

More info

(dolphin-env) //login4/scratch/09580/gdisilvestro/PacayaSenD26[1049] countbursts 
geom_reference/20141031   #of_bursts: 1   1
coreg_secondarys/20141124 #of_bursts: 0   0
coreg_secondarys/20141218 #of_bursts: 0   0
coreg_secondarys/20150429 #of_bursts: 0   0
coreg_secondarys/20150523 #of_bursts: 0   0
coreg_secondarys/20150616 #of_bursts: 0   0
coreg_secondarys/20150710 #of_bursts: 0   0
coreg_secondarys/20150827 #of_bursts: 0   0
coreg_secondarys/20150920 #of_bursts: 0   0
coreg_secondarys/20151014 #of_bursts: 0   0
coreg_secondarys/20151107 #of_bursts: 0   0
coreg_secondarys/20151201 #of_bursts: 0   0
coreg_secondarys/20151225 #of_bursts: 0   0
...

TEMPLATE

(dolphin-env) //login4/scratch/09580/gdisilvestro/PacayaSenD26[1046] cat /work2/09580/gdisilvestro/stampede2/insarlab/infiles/gdisilvestro/TEMPLATES/PacayaSenD26.template 
######################################################
ssaraopt.platform                  = SENTINEL-1  # [Sentinel-1 / ALOS2 / RADARSAT2 / TerraSAR-X / COSMO-Skymed]
ssaraopt.relativeOrbit             = 26
ssaraopt.startDate                 = 20141001  # YYYYMMDD
ssaraopt.endDate                   = auto  # YYYYMMDD
######################################################
minsar.excludeDates                = 20250501,20141020
######################################################
topsStack.subswath                 = 1 2 3 # '1 2'
topsStack.excludeDates             = 20250921
topsStack.numConnections           = 4    # comment
topsStack.azimuthLooks             = 4    # comment
topsStack.rangeLooks               = 16   # comment
topsStack.filtStrength             = 0.2  # comment
topsStack.unwMethod                = snaphu  # comment
topsStack.coregistration           = NESD  # [NESD geometry], auto for NESD
######################################################
mintpy.load.autoPath               = yes
mintpy.compute.cluster             = local #[local / slurm / pbs / lsf / none], auto for none, cluster type
mintpy.compute.numWorker           = 48 #[int > 1 / all], auto for 4 (local) or 40 (non-local), num of workers
mintpy.plot.maxMemory              = auto  #[float], auto for 4, max memory used by one call of view.py for plotting.
mintpy.networkInversion.parallel   = yes  #[yes / no], auto for no, parallel processing using dask
mintpy.save.hdfEos5                = yes   #[yes / update / no], auto for no, save timeseries to UNAVCO InSAR Archive format
mintpy.save.hdfEos5.update         = yes   #[yes / no], auto for no, put XXXXXXXX as endDate in output filename
mintpy.save.hdfEos5.subset         = yes   #[yes / no], auto for no, put subset range info in output filename
mintpy.save.kmz                    = yes   #[yes / no], auto for yes, save geocoded velocity to Google Earth KMZ file
mintpy.reference.minCoherence      = 0.7      #[0.0-1.0], auto for 0.85, minimum coherence for auto method
mintpy.troposphericDelay.method    = auto   # pyaps  #[pyaps / height_correlation / base_trop_cor / no], auto for pyaps
mintpy.networkInversion.minTempCoh = 0.6 #[0.0-1.0], auto for 0.7, min temporal coherence for mask
######################################################
miaplpy.load.processor               = isce
miaplpy.multiprocessing.numProcessor = 40
miaplpy.inversion.rangeWindow        = 24   # range window size for searching SHPs, auto for 15
miaplpy.inversion.azimuthWindow      = 7    # azimuth window size for searching SHPs, auto for 15
miaplpy.inversion.patchSize          = 150  # patch size (n*n) to divide the image for parallel processing, auto for 200
miaplpy.timeseries.tempCohType       = full     # [full, average], auto for full.
miaplpy.interferograms.networkType   = delaunay # network
miaplpy.unwrap.snaphu.tileNumPixels  = 10000000000     # number of pixels in a tile, auto for 10000000
######################################################
minsar.miaplpyDir.addition           = date  #[name / lalo / no] auto for no (miaply_$name_startDate_endDate))
miaplpy.subset.lalo                  = 14.335:14.399,-90.635:-90.562    #[S:N,W:E / no], auto for no
miaplpy.load.startDate               = auto  # 20200101
miaplpy.load.endDate                 = auto
mintpy.geocode.laloStep              = 0.0008,0.0008
miaplpy.timeseries.minTempCoh        = 0.75      # auto for 0.5
mintpy.networkInversion.minTempCoh   = 0.75
######################################################
minsar.insarmaps_flag                = True
minsar.upload_flag                   = True
minsar.insarmaps_dataset             = filt*DS

[falkamelung]

This is because NESD fails with noisy data.

As minsarApp.bash --help says, try:

minsarApp.bash  --delta-lat [DEG]        

to download larger error (default 0; bare --delta-lat: 0.2)

or

minsarApp.bash --coregistration geometry

head run_burst2stack
burst2stack --rel-orbit 26 --start-date 2015-04-29 --end-date 2015-04-30 --extent -90.64 14.34 -90.57 14.41 --keep-files --all-anns --pols VV 2> burst2stack_20150429.e
burst2stack --rel-orbit 26 --start-date 2015-05-23 --end-date 2015-05-24 --extent -90.64 14.34 -90.57 14.41 --keep-files --all-anns --pols VV 2> burst2stack_20150523.e
burst2stack --rel-orbit 26 --start-date 2015-06-16 --end-date 2015-06-17 --extent -90.64 14.34 -90.57 14.41 --keep-files --all-anns --pols VV 2> burst2stack_20150616.e
burst2stack --rel-orbit 26 --start-date 2015-07-10 --end-date 2015-07-11 --extent -90.64 14.34 -90.57 14.41 --keep-files --all-anns --pols VV 2> burst2stack_20150710.e
burst2stack --rel-orbit 26 --start-date 2015-08-27 --end-date 2015-08-28 --extent -90.64 14.34 -90.57 14.41 --keep-files --all-anns --pols VV 2> burst2stack_20150827.e
burst2stack --rel-orbit 26 --start-date 2015-09-20 --end-date 2015-09-21 --extent -90.64 14.34 -90.57 14.41 --keep-files --all-anns --pols VV 2> burst2stack_20150920.e
burst2stack --rel-orbit 26 --start-date 2015-10-14 --end-date 2015-10-15 --extent -90.64 14.34 -90.57 14.41 --keep-files --all-anns --pols VV 2> burst2stack_20151014.e
burst2stack --rel-orbit 26 --start-date 2015-11-07 --end-date 2015-11-08 --extent -90.64 14.34 -90.57 14.41 --keep-files --all-anns --pols VV 2> burst2stack_20151107.e
burst2stack --rel-orbit 26 --start-date 2015-12-01 --end-date 2015-12-02 --extent -90.64 14.34 -90.57 14.41 --keep-files --all-anns --pols VV 2> burst2stack_20151201.e
burst2stack --rel-orbit 26 --start-date 2015-12-25 --end-date 2015-12-26 --extent -90.64 14.34 -90.57 14.41 --keep-files --all-anns --pols VV 2> burst2stack_20151225.e
(minsar) //login3/scratch/05861/tg851601/PacayaSenD26/SLC[1457] burst2stack2vertex.bash burst2stack --rel-orbit 26 --start-date 2015-04-29 --end-date 2015-04-30 --extent -90.64 14.34 -90.57 14.41 --keep-files --all-anns
https://search.asf.alaska.edu/#/?zoom=9.1&center=-90.605,14.375&polygon=POLYGON((-90.64%2014.34,-90.57%2014.34,-90.57%2014.41,-90.64%2014.41,-90.64%2014.34))&dataset=SENTINEL-1%20BURSTS&maxResults=250&resultsLoaded=true&relativ
eOrbit=26&path=26-&start=2015-04-29T00:00:00Z&end=2015-04-30T23:59:59Z

minsarApp.bash 36.331:36.486,25.318:25.492 Santorini --coregistration geometry

Or just:

minsarApp.bash --geometry

[disilvestro]

#############################################################################################
20260520:11-14 * minsarApp.bash $TE/PacayaSenD26.template --start jobfiles --no-mintpy --geometry
Unknown option: --geometry
<lab/infiles/gdisilvestro/TEMPLATES/PacayaSenD26.template --start jobfiles --no-mintpy --coregistration geometry
sourcing /work2/09580/gdisilvestro/stampede3/code/minsar/minsar/bin/../lib/minsarApp_specifics.sh ...
sourcing /work2/09580/gdisilvestro/stampede3/code/minsar/minsar/bin/../lib/utils.sh ...
sourcing minsar/bin/../lib/utils.sh ...
#############################################################################################
20260520:11-14 * minsarApp.bash $TE/PacayaSenD26.template --start jobfiles --no-mintpy --coregistration geometry
Unknown option: --coregistration

[falkamelung]

Currently the options require an order. First the create_template.py options and then the minsarApp.bash options. So you have to run as:

minsarApp.bash $TE/PacayaSenD26.template --coregistration geometry --start jobfiles --no-mintpy

That allowed to implement this with minimal change to minsarApp.bash. I need to make it more flexible.

[disilvestro]

I am on main and I ran your command, still same error. The option doesn't exist

  Processing steps (start/end/dostep): 
   Command line options for steps processing with names are chosen from the following list: 
                                                                                 
   ['download', 'preprocess', 'dem', 'jobfiles', 'ifgram', 'mintpy', 'miaplpy']      
                                                                                 
   --upload    [--no-upload]    upload data products to jetstream (default)      
   --insarmaps [--no-insarmaps] ingest into insarmaps (default is yes for mintpy no for miaplpy)  
                                                                                 
   In order to use either --start or --dostep, it is necessary that a            
   previous run was done using one of the steps options to process at least      
   through the step immediately preceding the starting step of the current run.  
                                                                                 
   --start STEP          start processing at the named step [default: download]. 
   --end STEP, --stop STEP                                                       
   --dostep STEP         run processing at the named step only                   
   --download-method {slc, burst2safe, burst2stack, ssara-slc, ssara-bash, ssara-python} (default: burst2stack) 
   --delta-lat [DEG]     download larger error (default 0; bare --delta-lat: 0.2) 
                                                                                 
   --mintpy              use smallbaselineApp.py for time series [default]       
   --miaplpy             use miaplpyApp.py                                       
   --mintpy --miaplpy    use smallbaselineApp.py and miaplpyApp.py               
   --no-mintpy --miaplpy use only miaplpyApp.py                                  
                                                                                 
   --no-orbit-download   don't download orbits prior to jobfile creation         
   --opposite-orbit [--no-opposite-orbit]  run opposite-orbit step (default: off)       
   --horzvert [--no-horzvert]       run horzvert post-step after MiaplPy (default: off) 
                                                                                 
   --sleep SECS           sleep seconds before running                           
   --chunks               process in form of multiple chunks.                    
   --clean-start          rm -rf $SCRATCHDIR/<project> then recreate work dir   
                                                                                 
For sarvey:                                                                      
   --miaplpy-stop 1      to only run step 1 (load_slc_geometry.py) of miaplpy    
                                                                                 
Debug options:                                                                   
   --debug           sets set -x                                                 
   --skip-mintpy     skip mintpy processing (but runs everything else)           
   --skip-miaplpy    skip miaplpy processing (but runs everything else)          
   --start miaplpy --miaplpy-step 5

[falkamelung]

Sorry, if the template is already given, obviously it does not modify it. This is a create_template option that only works when a new template file is created:

minsarApp.bash 14.336:14.406,-90.639:-90.567 qPacaya --start-date 20150101 --flight-dir desc --coregistration geometry

[disilvestro]

I am getting the same error where, event though the area is very large, it says:
USER ERROR. No points left for reliable ESD estimate.Process additional bursts using --delta-lat 0.2. For islands use --coreegistration geometry (run_07_pairs_misreg_1_20251122_20251210_588.e)

I cannot use coregistration geometry otherwise it will process an even larger area.
I've never had this problem before since the template is the same I always had (I just changed the number of connection to test closure_phase_bias.py)

[disilvestro]

@falkamelung

[disilvestro]

Update: The area processed with coregistration: geometry can't be processed in slurm due to the extent of the area.

[disilvestro]

Furthermore i noticed that all the products now have extremely low coherence

[falkamelung]

coregistration: geometry processes the same area.
You may not have data as the download script could be buggy. Try burst2stack2vertex.bash

burst2stack2vertex.bash burst2stack --rel-orbit 87 --start-date 2025-01-01 --end-date 2025-12-31 --extent -86.6062 12.3516 -86.44 12.5019

[disilvestro]

this just prints an URL