Comparison with random forest

In this section, we will compare our small deep net to a Random Forest classifier. We will use the machine learning framework of the Orfeo ToolBox to train and evaluate the performance of the RF classifier.

Sample extraction¶

First, we use the exact same samples as for the deep net training and validation, to extract the pixels values of the image.

Use the SampleExtraction OTB application with the input samples locations vector layers to extract the input image pixel values and save them into vector layers:

part_2_sample_extraction_rf.py

import pyotb

# Input image (10m bands)
img = "/data/s2_tokyo_10m.tif"

# Input samples locations
pos_a = "/data/pos_a.geojson"
pos_b = "/data/pos_b.geojson"

# Output vector files
out_a = "/data/a_pixel_values.gpkg"
out_b = "/data/b_pixel_values.gpkg"

# Extract the pixels values at samples locations
for vec, out in zip([pos_a, pos_b], [out_a, out_b]):
    pyotb.SampleExtraction({"in": img, "vec": vec, "field": "class", "out": out})

Run the script to perform the sample extraction:

python part_2_sample_extraction_rf.py

After that, you should have two new vector data files created:

/data/a_pixel_values.gpkg
/data/b_pixel_values.gpkg

Question

Import the vector data files in QGIS,
Open their attribute table.

Training¶

Finally, we train a RF classifier with default parameters.

part_2_train_rf.py

import pyotb

# Output RF classifier
out = "/data/randomforest_model.yaml"

# Train a Random Forest classifier with validation
pyotb.TrainVectorClassifier(
    io_vd="/data/a_pixel_values.gpkg",
    valid_vd="/data/b_pixel_values.gpkg",
    feat=["value_0", "value_1", "value_2", "value_3"],
    cfield="class",
    classifier="rf",
    io_out=out,
)

Run the script:

python part_2_train_rf.py

Question

Note the F-Score of the different classes on the validation dataset,
Compare them with the metrics of the convolutional neural network.

The CNN has better performance on the validation dataset. To be sure, we should have compared on a third dataset (the test dataset) since this result is biased by the selection of the best model over the validation dataset (We will see how to do that in the next parts of this tutorial).

The CNN uses more information for training, compared to the RF. Indeed, it learns on 16x16patches of multispectral image, whereas the RF learns only on multispectral pixels (that could be interpreted as a 1x1 patches of multispectral image). Our goal here is not to compare RF versus CNN, but just to show that the contextual information process by the CNN improves the classification task result. A more fair competitor could be a RF using spatial features like textures, local Fourier transforms, SIFTs, etc.