Dear all,
we’re back for round two of PSI processing with StaMPS:
Define region of interest
A GeoJSON file (Simple Feature Access standard) describing the region of interest is needed. The easiest way to create a ROI polygon is by converting an existing Layer into a bounding box in QGIS (Vector / Research Tools / Extract Layer Extent) or by creating a new layer and manually mapping the ROI polygon:
- Layer / Create Layer / New Shapefile Layer
- Toggle Editing
- Edit / Add Feature
Either way, results can be exported as GeoJSON via Export / Save Features As / GeoJSON. Keep in mind to choose EPSG:4326 (WGS84) as CRS when exporting.
Get SENTINEL-1 data
Download through Open Access Hub
Downloading a bulk of Sentinel images, either S1 or S2, is not very smooth if you use the GUI from the Open Access Hub. However, the sentinelsat
Python package does provide a nice interface to the API:
from sentinelsat import SentinelAPI, read_geojson, geojson_to_wkt
api = SentinelAPI('username', 'password')
footprint = geojson_to_wkt(read_geojson('/path/to/roi.geojson'))
products = api.query(footprint,
platformname='Sentinel-1',
producttype='SLC',
orbitdirection='DESCENDING',
beginposition='[YYYY-MM-DDT00:00:00.000Z TO YYYY-MM-DDT00:00:00.000Z]',
sensoroperationalmode='IW',
polarisationmode='VV')
api.download_all(products, directory_path='/path/to/directory')
To see all options you can use to communicate with the server have a look at the sentinelsat documentation and the original API documentation of the data hub.
snap2stamps
General Information
This chapter is directly taken from the StaMPS-manual and the snap2stamps-manual.
StaMPS is compatible with the output generated by the ESA SentiNel Application Platform (SNAP) after the version 6.0. SNAP allows the user to define a series of xml files which contain user defined processing workflow by using its Graph Builder. These files can be used to run SNAP processing in batch mode by using the GPT command (Graph Processing Tool).
snap2stamps
contains a set of graphs, together with python wrappers that allow you to automatise the interferogram processing chain for single master interferograms compatible with StaMPS PSI. Information about the provided functionalities and their instructions can be found in the user manual provided within snap2stamps, which already plans newer releases increasing functionality and compatibility maintenance of SNAP-StaMPS chain. A reference to the software package (Foumelis et al., 2018) can be found in the manual.
The following steps are needed and provided by snap2stamps:
- Sort the slave images in separated folders corresponding to their acquisition time.
- Slave splitting (and slice assembling if needed) and annotated orbits replacement with Precise/Restituted Orbits.
- Coregistration and interferogram generation done per slave and subswath. Here auxiliary files as DEMs and subset using a bounding box is performed. Both amplitude coregistered master-slave stack and interferogram with elevation and orthorectified latitude and longitude coordinates are saved as output.
- StaMPS export is done providing for each pair coregistered stack and interferogram generated StaMPS compatible products.
snap2stamps contains the following files:
project.conf
– file with parameters and paths needed for the processingslaves_prep.py
– script for sorting slaves into the expected folder structuresplitting_slaves_logging.py
– script for slave splitting (and assembling if needed) and orbit correctioncoreg_ifg_topsar.py
– script for master-slave coregistration and interferometric generationstamps_export.py
– script for ouput data generation in StaMPS compatible format for PSI processing
Note that snap2stamps requires Python 2.7.
snap2stamps Workflow
- Select optimal master in SNAP using Radar / Interferometric / InSAR Stack Overview
- Perform subsetting of whole image using TOPSAR Split via Radar / Sentinel-1 TOPS / S-1 TOPS Split. Set the processing parameters
- subswath (IW 1-3),
- polarization (vertical sent, vertical returned – VV), and
- bursts (using the bursts-slider)
in the respective tab.
- Get LAT/LON MIN/MAX (bounding box) for PSI area of interest.
This can be obtained e.g. from ROI polygon in QGIS (Layer Properties / Metadata / Properties / Extent). - (To keep a backup, you might want to simply copy your snap2stamps
bin
andgraphs
folders to your working directory). Editproject.conf
to set up the configuration for your project. An example configuration file might look like this:
######### CONFIGURATION FILE ######
###################################
# PROJECT DEFINITION
PROJECTFOLDER=/home/user/.../project
GRAPHSFOLDER=/home/user/.../project/graphs
##################################
# PROCESSING PARAMETERS
IW1=IW2
MASTER=/home/user/.../master/S1B_IW_SLC__1SDV_..._split.dim
##################################
# AOI BBOX DEFINITION
LONMIN=14.xx
LATMIN=47.xx
LONMAX=14.xx
LATMAX=47.xx
##################################
# SNAP GPT
GPTBIN_PATH=/usr/local/snap/bin/gpt
##################################
# COMPUTING RESOURCES TO EMPLOY
CPU=12
CACHE=32G
##################################
- Move the master (zip + TOPS – Split Output) to the directory
master
in your PROJECTFOLDER. - Make sure that all slave images (zip) are stored in the subfolder
slaves
in the PROJECTFOLDER. - Check if all libraries are available for your Python 2 installation (you might need to
pip install pathlib
). I strongly recommended to use a corresponding conda environment for Python 2.7:conda activate snap2stamps
. - Run the python scripts of snap2stamp directly in your shell:
# slave sorting
# (fast)
python slaves_prep.py project.conf
# slave splitting and orbit correction
# (approx. 50 seconds per slave)
python splitting_slaves.py project.conf
# master-slave coregistration and interferometric generation
# (approx. 180 seconds per slave)
python coreg_ifg_topsar.py project.conf
# ouput data generation in StaMPS compatible format
# (approx. 30 seconds)
python stamps_export.py project.conf
- The data final output structure after performing the last step should contain these four folders:
rslc
,diff0
,geo
anddem
within yourINSAR_master_date
folder. - Check for empty interferograms. If any exist, remove files containing the date of the empty file from the folders
rslc
anddiff0
. Otherwise, this will throw warnings related to 0 mean amplitude during the final preparation step use in stamps (i.e.mt_prep_snap
) and eventually screw up the selection of PS candidates. - Prepare the data set for final use in StaMPS using
mt_prep_snap
with<yyyymmdd>
being the master date. The path after the date is the full filepath to theINSAR_master_date
folder:
mt_prep_snap /home/username/…/INSAR_master_date/ DA
- The amplitude dispersion index
is a value that describes the amplitude stability, which is used to preselect pixels and therefore reduces the number of pixels for the phase analysis. It is calculated as
, where
is the standard deviation and
is the mean of a series of amplitude values.
- The recommended range for
is 0.4-0.42. The higher the threshold, more pixels will be selected for phase analysis. Note that surfaces like water and vegetation, where amplitude is instable, exhibit higher
values than bedrock outcroppings or man-made strucutres (i.e. most likeley PS pixels). See Ferretti et al. (2001) and Hooper et al. (2007) for further information. Additional arguments for
mt_prep_snap
are only needed for large areas (i.e. if multiple patches are processed). - Finally, launch
matlab
to continue with StaMPS PS analysis.
This is the basic processing chain for obtaining Sentinel-1 SLC data and preparing them for further use in StaMPS.
5 Comments
You can post comments in this post.
Hi, I am working on a subject for my university and i just saw your post. I was wondering if you can help me with few questions.
I´m trying to run the process in a zone of Colombia (a Sentinel image over 2 Burst big).
following your step by step process (on windows 10 with 4 cpus and 20 GB).
I´m am looking at those time steps that you just put on the preparation of data for Stamps.
At the code “python coreg_ifg_topsar.py project.conf” and since i have less machine power than you, I expect the process to take longer than yours. But it actually is taking so long (half a day without ending the process).
Do you think I should change something or is because the operative system or something like that
Camilo De La Hoz Lozano 4 years ago
Hi,
the master-slave coregistration and interferometric generation is indeed the most time consuming preprocessing step. Time needed to run the script depends on the and the length of the time series (i.e. the number of images), the size of the area and computing performance. If you use for instance both S-1A and S1-B imagery over the full time period available (2016-2019), this will indeed take some time on a slow computer. However, more than 12 hours does indeed seems suspiciously long. Unfortunately, there is not much that you can tweak, except for the bounding box for your AOI and the CPU resources employed.
Matthias 4 years ago
the problem was in the place in where you define the CACHE and CPU.
I did not manage to fix it. But, I put it as a comment and it worked fine (took me 25 min per slave). I am still trying to solve why this happens. I believe it is related to the operative system Windows
Camilo De La Hoz Lozano 3 years ago
Greetings, I am doing a study of subsidence in the dam of a dam, the high levels are areas without vegetation cover, that is to say they are surface to the interperie where the band C of Sentinel, I consider that it can determine subsidence and in a part of the work civil of the dam, but I’m still in the phase of developing the phase with Snaphu, in that sense I have the following questions:
How accurate are the data obtained from snaphu, and how much they vary when applying the PSI or SBAS technique. and If you currently do not have PSI processing videos with SNAP Stamps.
Oliver 4 years ago
can you make a tutorial video, because i am new to linux.it would be very helpful
Kazi Hifajat 3 years ago
Post A Reply