A more complex processor

In this chapter we explain how to process products from the catalogue. The example will be the "water mask" processor which creates a binary land/water mask. For this part we will need to use an Ubuntu console, as explained in the Interactive Application tutorial. In the Ubuntu console, open a terminal (applications menu) and copy /shared/CTEP-TUTORIAL/water_mask.py in your user Basket. Check that everyone has execution rights on this file and chmod +rx if necessary.

Let's have a look at this file. You may install your favorite editor for this. For instance apt-get install gedit, then gedit water_mask.py &. The python script can be run directly on the CTEP, if it starts with #!/bin/python. Many python libraries are already installed on the CTEP system, contact us if you need something else. Here we use the sys.argv command to retrieve the input data.

The script ends with the creation of the water mask in Geotiff format. The CTEP will automatically recognize Geotiff files created by your processor and propose them for visualization. The file shall be created in the CTEP_OUTPUT_DIR folder whose value can be retrieved with a os.getenv command. The default value . ensures that the script will also run normally outside of the CTEP environment.

We will now upload the processor from within the Ubuntu console. Open a browser in your ubuntu session and log-in again to the CTEP geobrowser. From the Processor Integration page, you can define a new processor. The input product shall have the type "file". This specific type activates the connection with the Input Data clipboard.

Once you have defined the inputs, specify the processor command (water_mask.py) in the Execution tab. Then upload the tar.gz file with the processor. After integration, the processor is available in the processor list.

After execution, the new geotiff file can be published on the browser ("menu" icon beside the job result). Remember that you can check the output log by clicking on the info menu ("i" icon). All files with a .tif extension will be available for visualisation.

<img src="https://gitlab.acri-cwa.fr/coastal-tep/tutorial/raw/master/images/part5/publish_layer.jpg" width="50%" style:"float:left;"><img src="https://gitlab.acri-cwa.fr/coastal-tep/tutorial/raw/master/images/part5/venice_mask.jpg" width="50%" style:"float:right;">

Accessing files

All input files will be placed by the CTEP in the working directory before execution. If needed, the path to this directory can be accessed with the environment variable CTEP_WORK_DIR.

The files of the processor package are installed in /usr/local/ctep-user-processor/. Use this path to read configuration files for instance. This folder is included by default in the PATH and PYTHONPATH environment variables.