If you’ve been trying to come here over the past few days, you might have noticed that this blog has been up and down, changing themes, and what not. I have been having issues upgrading the PHP version on this website and finally got things ironed out thanks to my provider’s awesome support staff! So I promise it should be back to normal now. Mostly. Probably. đ
Category Archives: Uncategorized
Image Processing Basics Part 2
Some Examples
Now that we have some of the basics down, let us look at some practical examples of the differences between how the brain sees things versus how a computer does.
The above photo of a part of the sky was taken by my iPhone 13 Pro Max using the native camera application. There were no filters or anything else applied to it. To our eyes, it looks fairly uniform: mainly blue with some lighter blue towards the right where the sun was the day I took the picture. Each pixel of the image represents the light that hit a sensor in the camera, was processed, and saved.
Our brain does not see a number of individual pixels. Instead, we see large splotches of colors. This is one of the shortcuts our brain does to ease the processing burden. If you look around a room, you do not see individual differences between the colors of the wall. Your wall mainly looks like a uniform color. We simply do not have the processing power to break down the inputs from our eyes into every minute part.
A computer, however, does have the ability to “see” an image in all of its different parts. Computers see everything as a number, be it the 1âs and 0âs of binary or color triplets in the RGB color space. If we look at the RGB color cube below, the computer sees all of the pixels in the above image as clustering somewhere around the lower right side of the cube. See the previous link for more information about the RGB color space.
In a computer, the above image is loaded and each pixel is in memory in the form of triplets such as (135, 206, 235), which is the code for a color known as sky blue. The computer also does not have to take any shortcuts when it loads the image, meaning that the representation in memory is exactly the same as the image that was saved from the phone.
If we use the OpenCV library to calculate the histogram of the image and then count the number of colors, we in fact find that there are 2,522 unique colors in the picture of the sky. There is no magic here, we just do not have the same precision that a computer does when it comes to examining images or our environment. The big take away here is this: there is more information encoded in pictures or video than what our brains are capable of perceiving. Just because we cannot see certain details in a image does not mean that they are not there.
For another example, consider this image below. The edges look like nothing but black, and all you can really see is out of the window. It is definitely underexposed.
As mentioned above, a computer is able to detect more than our eyes can. Where we just see black around the edges, there is in fact detail there. We can adjust the exposure on the image to brighten it so that our eyes can see these details.
With the exposure turned up (and adjusting the contrast as well), we can additionally see a picture of a bird, some dishes, and some cooking implements. This is not magic, nor is it adding anything to the image that was not already there. Image processing like this does not insert things into an image. It only enhances the details of an image so that they are more detectable to the human eye.
Many times, when image processing is in the news, people sometimes assume that it changing an image, or that it is inserting things that were not originally there. When you edit your images on your phone or tablet, you are manipulating the detail that is already in the image. You can enhance the contrast to make the image “pop.” You can change the color tone of the image to make it appear more warm or more cold to your liking. However, this is simply modifying the information that is already in the image to change how it appears to the human eye.
I am making a big deal about this point as future installments in this series will demonstrate how things actually work while hopefully dispelling certain myths that exist in pop culture. I think next time I will cover zooming in or out of an image (aka, resizing). Does it add something into the image or misrepresent it? We will find out.
When Checkinstall Attacks
The other day I was compiling the latest OpenCV on my computer and had planned on doing what I normally do when itâs done: run checkinstall to build a .deb for it because I like to keep all my files under package management. OpenCV finished compiling fairly quickly (itâs nice when you can do a make -j 16) and I then ran checkinstall.
It crashed while it was running and left a half-installed Debian package of OpenCV on my system. âNo problemâ I thought, Iâll just uninstall the deb and do a normal make install. Sometimes checkinstall crashes so I didnât think anything was out of the ordinary. Since I usually put it in /opt/opencv4 it would still be self contained at least.
I noticed a little bit later that my system was acting oddly. Some things wouldnât run, I couldnât sudo any more, etc. I rebooted as a first check to see if it was just something random going on. And thatâs when my system rebooted to a text mode login prompt. âHuh, maybe the card/drivers didnât initialize fully Iâll just reboot again.â Nope, no joy still the text login.
I tried to login only to watch the process pause after I typed my password, and then came back up the login prompt. âOdd, maybe Iâll see if itâs something weird and try another virtual console.â Nope, no joy there. Tried to ssh into it, no joy there either. I was worried my SSD was going out. Itâs not that old, but still a worry.
So I used my laptop to make a bootable Mint installer and plugged that in and tried to boot. The graphics screen was corrupted and had to use safe mode to log in. âHoly crap, is my graphics card messed up along with the hard drive?â I was worried about this because a new power supply I bought a while back had nuked my old motherboard so had to replace hardware in my system. (Thatâs a story for another day).
I could still get a GUI when I booted into safe mode from the thumb drive so assumed the open source drivers on the latest Mint installer just didnât like my card unless I did safe mode.. I did a SMART test to make sure nothing was wrong with the drive. That worked so I ran a fsck to check the integrity of the drive. I then went to set up a chroot to the hard drive so I could run debsums to make sure the packages hadnât gotten randomly corrupted. And then I noticed a problem.
I couldnât set up the chroot to work. I kept getting an error about /bin/bash not existing. I checked the /bin directory on the hard drive and sure enough, it was empty save for a broken link to some part of the JDK. âThatâs odd, there were no drive errors but /bin is empty.â I thought about things for a moment and it randomly did an ls -ld on the root of the hard drive but didnât see anything at first.
Then hit it me: âWait a minute, /bin is supposed to be a link to /usr/bin these days.â I realized that for whatever reason, it looked like checkinstall had replaced the link for /bin with an actual /bin and had randomly placed a link in there for the jdk. I deleted the directory and replaced the link to /usr/bin and rebooted. Boom, system booted normally. Well, mostly normally. CUDA had somehow disappeared from the drive and I had to reinstall it (didnât use the packages from nVidia since they want to downgrade my video drivers so just did a local install). I ran debsums to check and everything verified properly.
The moral of the story is, itâs good to have debugging skills and know how your computer is supposed to work!
National/State-Level Census Tiger files updated
Hot off the computer processing over the weekend, and as a historic first for me only a few days after the Census released them, I’ve updated my GIS data to the 2021 Census Tiger files. There are combined to either the state or national level.
Brian vs the Inspiron 620S
On Memorial Day I can say I had a memorable experience while trying to troubleshoot an old computer we still use. My wife got a Dell Inspiron 620S a while ago to use for her work and what not. Over the years I put a bigger hard drive in it and upgraded it to Windows 10. Itâs not the fastest computer, but it still works for my wifeâs vinyl cutter program that she uses and some software her work uses thatâs Windows only. My kids also periodically use it for older games that they like to play since itâs a Core i5 with a decent low-end Radeon card in it.
A few weeks ago it just stopped working. It would not turn on even though the power supply LED was on and the power LED on the motherboard was lit. Just nothing would happen when you pushed the power button. No hard drive spin ups, nothing. So I let it sit for a while.
On Memorial Day I thought I would finally see what was up with it. I took out my multimeter because my first thought was perhaps the power supply was old and wasnât producing enough power. I checked the ATX motherboard connector and the always-on pin had power and was the right voltage. I also inspected the motherboard to see if perhaps any capacitors had blown but everything looked fine.
I got up in frustration and thought Iâd look online. As I got up, my foot came down on something and then slipped which did bad things to my toes and the muscles/ligaments in my foot and also came close to cutting off my middle toe. I thought it was actually a bit fitting because it was like the computer had found a way to flip me off đ Thankfully my wife has been helping me keep my toes buddy taped together and have been keeping stuff on the cut to make sure it heals.
I did finally do some Internet searching and found several other owners complaining on the Dell and other forums about the same issue. They had found that unplugging the two front USB ports from the motherboard fixed their issue. No one has any idea what could be going on and of course I havenât found anything from Dell admitting to it.
On Saturday we decided to take the kids for ice cream (my wife has been driving lately since itâs hard for me to walk let alone drive). I thought before we left Iâd hobble down and just see if unplugging those ports would work. I went to the computer (after making sure there was nothing on the floor around the desk), found the ports on the motherboard and unplugged them. And of course, the computer magically started turning on again. I still donât know why unplugging them works, but Iâm also not going to argue.
I then ran into my next problem. It had been a while since anyone used that computer since it hadnât been working for several weeks and it wasnât a priority. I randomly could not log in. My password didnât work, none of the normal passwords I use around the house worked, no joy. My wife and daughter could log in but I couldnât. I think it was the angry computer gods giving me one last middle finger.
Fortunately I keep a multi-boot USB handy that has a lot of bootable distributions and utilities. I booted into Kali Linux forensics mode. Once booted all I had to do was open up a terminal, switch to the directory on the Windows drive that had the SAM files, and was able to use chntpw to blank out my password. While there I made sure my account hadnât been locked out or anything like that. I also booted into several antivirus tools and scanned the hard drive just in case there was a reason my password wasnât working and did a chkdisk on the drive to make repair the errors from when it stopped working after the power outage.
Things are back to normal with it now and Iâve gotten the updates done that it had been missing. I think my next step is a Catholic priest and some Holy Water just in case!
State-Based 2020 Census Tiger Data Out
Yes, it’s April 2021, and I just now got around to repackaging the 2020 Census Tiger Data at state/national levels. If you’re interested, head over to here.
More Fun with the RTX 2060
So I recently wiped my system and upgraded to Linux Mint Cinnamon 20. I tend to wipe and install on major releases since I do a lot of customization.
Anyway, I wanted to set CUDA back up along with tensorflow-gpu since I have stuff I wanted to do. I recreated my virtual environment and found Tensorflow 2.2.0 had been released. Based on this I found it still needs CUDA 10.1. No worries, went through and put CUDA 10.1, cuDNN, and TensorRT back on my system and everything was working.
I noticed with 2.2.0 that I was getting the dreaded RTX CUDA_ERROR_OUT_OF_MEMORY errors for pretty much anything I did. So I fixed it and figured I’d post this in case it helps anyone else out down the road. You need to add this in so that the GPU memory can grow and use mixed precision with the RTX (which also helps to run things on the TPUs in the RTX series).
from tensorflow import config as tfc
from tensorflow.keras.mixed_precision import experimental as mixed_precision
...
gpus = tfc.experimental.list_physical_devices("GPU")
tfc.experimental.set_memory_growth(gpus[0], True)
policy = mixed_precision.Policy('mixed_float16')
mixed_precision.set_policy(policy)
If you’re having more out of memory errors on your RTX, give this a shot. You can read more about Tensorflow and mixed precision here.
Fun with Linux and a RTX 2060
Or…. how to spend a day hitting your head into your desk.
Or…. machine learning is easy, right? đ
For a while now I have been wanting to upgrade my video card in my desktop so I could actually use it to do machine/deep learning tasks (ML). Since I put together a Frankenstein gaming computer for my daughters out of some older parts, I finally justified getting a new card by saying I would then give them my older nVidia card. After a lot of research, I decided the RTX 2060 was a good balance of how much money I felt like spending versus something that would actually be useful (plus the series comes with dedicated Tensor Cores that work really fast with fp16 data types).
So after buying the card and installing it, the first thing I wanted to do was to get Tensorflow 2.1 to work with it. Now, I already had CUDA 10.2 and the most up-to-date version of TensorRT installed, and knew that I’d have to custom compile Tensorflow’s pip version to work on my system. What I did not know was just how annoying this would turn out to be. My first attempt was to follow the instructions from the Tensorflow web site, including applying this patch that fixes the nccl bindings to work with CUDA 10.2.
However, all of my attempts failed. I had random compiler errors crop up during the build that I have never had before and could not explain. Tried building it with Clang. Tried different versions of GCC. Considered building with a Catholic priest present to keep the demons at bay. No dice. Never could complete a build successfully on my system. This was a bit to be expected since a lot of people online have trouble getting Tensorflow 2.1 and CUDA 10.2 to play nice together.
I finally broke down and downgraded CUDA on my system to 10.1. I also downgraded TensorRT so it would be compatible with the version of CUDA I now had. Finally I could do a pip install tensorflow-gpu inside my virtual environment and and it worked and it ran and I could finally run my training on my GPU with fantastic results.
Almost.
I kept getting CUDNN_STATUS_INTERNAL_ERROR messages every time I tried to run a Keras application on the GPU. Yay. After some Googling, I found this link and apparently there’s an issue with Tensorflow and the RTX line. To fix it, you have to add this to your Python code that uses Keras/Tensorflow:
from tensorflow.compat.v1 import ConfigProto
from tensorflow.compat.v1 import InteractiveSession
...
config = ConfigProto()
config.gpu_options.allow_growth = True
session = InteractiveSession(config=config)
FINALLY! After several days of trying to custom compile Tensorflow for my system, giving up, downgrading so I could install via pip, running into more errors, etc, I now have a working GPU-accelerated Tensorflow! As an example, running the simple additionrnn.py example from Keras, I went from taking around 3.5 seconds per epoch on my Ryzen 7 2700X processor (where I had compiled Tensorflow for CPU only to take advantage of the additional CPU instructions) to taking under 0.5 seconds on the GPU. I’m still experimenting, and modifying some things to use fp16 so I can take advantage of the Tensor Cores in the GPU.
Tiger 2019 Data Uploaded
I’ve uploaded my state-based versions of the Census Tiger 2019 datasets. Use at your own risk, yadda yadda. See the GIS Data page above.
Revisiting Historic Topographic Maps Part 2
Reply
In part one I discussed how I go about finding and downloading maps from the USGS Historic Topomap site. Now I will show how I go through the images I downloaded in Part 1 and determine which ones to keep to make a merged state map. I will also show some things you may run into while making such a map.
Now that the maps are all downloaded, it is time to go through and examine each one to determine what to keep and what to digitally âthrow out.â When you download all the historic maps of a certain scale for a state, you will find that each geographic area may have multiple versions that cover it. You will also find that there are some specially made maps that go against the standard quadrangle area and naming convention. The easiest way for me to handle this is to load and examine all the maps inside QGIS.
Using QGIS to Check an Image
I load the maps that cover a quadrangle and overlay them on top of something like Google Maps. For my purposes, I usually try pick maps with the following characteristics:
- Oldest to cover an area.
- Good visual quality (easy to read, paper map does was not ripped, etc)
- Good georeferencing to existing features
QGIS makes it easy to look at all the maps that cover an area. I will typically change the opacity of a layer and see how features such as rivers match existing ones. You will be hard-pressed to find an exact match as some scales are too coarse and these old maps will never match the precision of modern digital ones made from GPS. I also make sure that the map is not too dark and that the text is easily readable.
One thing you will notice with the maps is that names change over time. An example of this is below, where in one map a feature is called Bullockâs Neck and in another it is Bullitt Neck.
Feature Named Bullock’s Neck
Feature Named Bullitt Neck
Another thing you will find with these maps is that the same features are not always in the same spots. Consider the next three images here that cover the same area.
Geographic Area to Check for Registration
First Historic Map to Check Registration
Second Map to Check Registration
If you look closely, you will see that the land features of the map seem to move down between the second and third images. This happens due to how the maps were printed âback in the day.â The maps were broken down into separates where each separate (or plate) contained features of the same color. One contained roads, text, and houses, while another had features such as forests. These separates had stud holes in them so they could be held in place during the printing process. Each separate was inked and a piece of paper was run over each one. Over time these stud holes would get worn so the one or more would move around during printing. Additionally, maps back then were somewhat âworks of art,â and could differ between who did the inscribing. Finally, depending on scale and quality of the map, the algorithms to georeference the scanned images can result in rubber sheeting that can further change things.
During my processing, one of the things I use QGIS for is to check which maps register better against modern features. It takes a while using this method but in the end I am typically much happier about the final quality than if I just picked one from each batch at random.
Another thing to check with the historic maps is coverage. Sometimes the map may say it covers a part of the state when it does not.
Map Showing No Actual Virginia Coverage
Here the map showed up in the results list for Virginia, but you can see that the Virginia portion is blank and it actually only contains map information for Maryland.
Finally, you may well find that you do not have images that cover the entire state you are interested in. If you group things by scale and year, the USGS may no longer have the original topomaps for some areas. It could also be that no maps were actually produced for those areas.
Once the images are all selected, the images need to be merged into a single map for an individual state. For my setup I have found that things are easier and faster if I merge them into a GeoTIFF with multiple zoom layers as opposed to storing in PostGIS.
Here I will assume there is a directory of 250K scale files that cover Virginia and that these files have been sorted and the best selected. The first part of this is to merge the individual files into a single file with the command:
gdal_merge.py -o va_250k.tif *.tif
This command may take some time based on various factors. Once finished, the next part is to compress and tile the image. Tiling breaks the image into various parts that can be separately accessed and displayed without having to process the rest of the image. Compression can make a huge difference in file sizes. I did some experimenting and found that a JPEG compression quality of eighty strikes a good balance between being visually pleasing and reducing file size.
gdal_translate -co COMPRESS=JPEG -co TILED=YES -co JPEG_QUALITY=80 va_100k.tif va_100k_tiled.tif
Finally, GeoTIFFs can have reduced-resolution overview layers added to them. The TIFF format supports multiple pages in a file as one of the original uses was to store faxes. A GIS such as QGIS can recognize when a file has overlay views added and will use them first based on how far the user has zoomed. These views usually have much fewer data than the full file and can be quickly accessed and displayed.
gdaladdo --config COMPRESS_OVERVIEW JPEG --config INTERLEAVE_OVERVIEW PIXEL -r average va_100k_tiled.tif 2 4 8 16
With the above command, GDAL will add an overview that is roughly half sized, quarter sized, and so on.
In the end, with tiling and compression, my 250K scale merged map of Virginia comes in at 520 megabytes. QGIS recognizes that the multiple TIFF pages are the various overviews and over my home network loading and zooming is nearly instantaneous. Hopefully these posts will help you to create your own mosaics of historic or even more modern maps.