{"id":723,"date":"2024-02-10T14:06:53","date_gmt":"2024-02-10T19:06:53","guid":{"rendered":"https:\/\/brian.digitalmaddox.com\/blog\/?p=723"},"modified":"2024-02-10T14:06:53","modified_gmt":"2024-02-10T19:06:53","slug":"applying-deep-learning-and-computer-vision-to-lidar-part-2-training-data","status":"publish","type":"post","link":"https:\/\/brian.digitalmaddox.com\/blog\/?p=723","title":{"rendered":"Applying Deep Learning and Computer Vision to Lidar Part 2: Training Data"},"content":{"rendered":"\n

In part one<\/a> I described some of the issues I had on a recent project that applied deep learning to geographic feature recognition in LiDAR and the file sizes of such data.\u00a0 This time I want to talk about training data, how important it is, and how little there is for this type of problem.<\/p>\n\n\n\n

One of the most important things in deep learning is having both quality training data and a good amount of such data.\u00a0 I have actually written a previous post<\/a> about the importance of quality data that you can read here.\u00a0 At a bare minimum, you should typically have around one thousand samples of each object class you want to train a model to recognize.\u00a0 Object classes in this case are geographic feature types that we want to recognize.<\/p>\n\n\n\n

Training Data Characteristics<\/h2>\n\n\n\n

These samples should mirror the characteristics of the data that your model will come across during classification tasks.\u00a0 With LiDAR in GeoTIFF format, the training data should be similar in resolution (0.5 meters in this case) and bit depth (32-bit) to the area for testing.\u00a0 There should be variability in the training data.\u00a0 Convolutional neural networks are NOT rotational invariant<\/a>, meaning that unless you train your model on samples at different angles, it will not automatically recognize features.\u00a0 In this case, your training LiDAR features should be rotated at different angles to account for differences in projection or north direction.<\/p>\n\n\n\n

Balanced Numbers of Features per Class<\/h2>\n\n\n\n

Your training data should also be balanced, meaning that each class should have roughly the same number of training images where possible.\u00a0 You can perform some tasks that we will talk about in a bit to help with this, but generally if your model is unbalanced, it could mistakenly \u201clean\u201d towards one class more than others.<\/p>\n\n\n\n

Related to this, when generating your training data, you should make sure your training and testing data contain samples from each of your object classes. Especially when imbalanced, it is very easy to use something like train_test_split<\/a> from the sklearn<\/a> library and have it generate a training set that misses some object classes. You should also shuffle your training data so that the samples from each class are sufficiently randomized and the model does not assume that features will appear in a specific order. To alleviate this, make sure you pass in something like:<\/p>\n\n\n\n

\u2026 = train_test_split(\u2026, shuffle=True, stratify=labels)<\/code><\/pre>\n\n\n\n
where labels<\/strong> is a list of your object classes.  For most cases this will ensure the order of your samples is sufficiently randomized and that your training\/testing data contains samples of each object class.<\/p>\n\n\n\n
Complexity<\/h2>\n\n\n\n
Geographic features also have a varying complexity in how they appear in LiDAR.\u00a0 The same feature can \u201clook\u201d differently based on its size or other characteristics.\u00a0 Humans will always look like humans, so training on them is fairly simple.\u00a0 Geographic features can be in the same class but appear differently based on factors such as weathering, erosion, vegetation, and even if it\u2019s a riverbed that is dry part of the year.\u00a0 This increased complexity means that samples need to have enough variability, even in the same object class, for proper training.<\/p>\n\n\n\n
Lack of Training Data for This Project<\/h2>\n\n\n\n
With all of this out of the way, let us now talk about the issues that faced this particular project.\u00a0 First of all, there is not a lot of labeled training data for these types of features.\u00a0 At all.\u00a0 I used various search engines, ChatGPT, even bought a bucket of KFC so I could try to throw some bones to lead me to training data (although I don\u2019t know voodoo so probably read them wrong).<\/p>\n\n\n\n
There is a lot of data about these geographic features out there, but not labeled AND<\/strong> in LiDAR format.\u00a0 There is an abundance of photographs of these features.\u00a0 There are paper maps of these features.\u00a0 There are research papers with drawings of these features.\u00a0 I even found some GIS data that had polygons of these features, but the matching elevation model was too low of a resolution to be useful.<\/p>\n\n\n\n
In the end I was only able to find a single<\/strong> dataset that matched the bit depth and spatial resolution that matched the test data.\u00a0 There were a couple of problems with this dataset though.\u00a0 First, it only had three feature classes out of a dozen or more.\u00a0 Second, the number of samples of each of these three classes were way imbalanced.\u00a0 It broke down like this:<\/p>\n\n\n\n