Understanding pictures and drawing by the visually challenged

Richard Baldwin is a Professor of Computer Information Technology at Austin Community College in Austin, Texas, USA.

Prof Baldwin first became involved in the issues confronting blind students when a blind-from-birth student named Amanda Lacy enrolled in an object-oriented programming course that he taught. Ms. Lacy successfully completed that course and two others that he taught. Knowing that she was a very good student, he became concerned upon learning that she was having difficulty in a physics course. Being an engineer by training, he volunteered to tutor her in the physics course. This led to the two of them spending about four hours per week in tutoring sessions for the past couple of years.

Prof. Baldwin quickly learned that neither the physics lectures nor the physics textbook were accessible to Ms. Lacy. She had stopped attending lectures because she was unable to follow along. The electronic copy of the textbook that she had been provided contained hundreds of equations and pictures that were incompatible with her assistive tools: a screen reader and a Braille pad.

Prof. Baldwin began by authoring an online physics tutorial titled “Accessible Physics Concepts for Blind Students” (available here) to help Ms. Lacy through the first semester physics course. Although it took more than one semester for her to complete the course, she did complete it and made an A in the course.

While authoring the online physics tutorial, Prof. Baldwin came to realize that unlike sighted students, blind students were unable to draw and submit graphical homework assignments, such as vector diagrams with any accuracy. Although there are several outstanding free drawing programs available on the web, all or at least most of them require the user to manipulate a mouse. Therefore, they are not accessible to blind students. As a result, Prof. Baldwin wrote a drawing program named SVGDraw01 that is fully accessible for blind students. The theme for this program is “if you can imagine it, you can draw it.” The program is freely available for downloading here.

Later, Prof. Baldwin decided to tackle the inaccessible pictures and diagrams in textbooks. One technique that blind students can use to understand the content of diagrams and pictures is to emboss them using any of several available embossing techniques. Basically, embossing an image means to create a tactile document containing raised lines or raised dots that describe the salient features of the image.

Originally, Prof. Baldwin hoped to make it possible for blind students to emboss their own textbook images. So far, that is still a dream for several reasons. The most common format of electronic textbook is the Adobe pdf format. Without the purchase of expensive software (and probably not even then), it is impossible for a blind student to successfully extract most of the images in a pdf file intact. While there are numerous programs available that claim to extract the images from pdf files, in most cases each image ends up in several different files that must be reassembled for embossing.

After coming to that realization, Prof. Baldwin decided to direct his attention to the many technical issues that make it very difficult for even a sighted person to convert a typical bitmap file into an embossed image that is meaningful to a blind student.

There are about 16-million good reasons why the embossed version of a full-color bitmap image often fails to produce satisfactory tactile results. Begin with the fact that the embossing process often discards the information content from more than 16-million colors ending up with what is effectively black and white, or possibly black and white with two or three shades of gray in between, depending on the embossing method.

Follow the loss of color information with the fact that, unless the original image is very small, the spatial sampling is probably reduced by a factor of 5 to 10 in the embossed image with little or no consideration being given to frequency aliasing that occurs in a brute-force spatial re-sampling process.

The bottom line is that it is very difficult to emboss full-color bitmap images and end up with high-quality tactile images.

Different embossing methods produce different physical outputs. Many of the older Braille printers have a graphics mode that allows pictures to be displayed by raising a subset of individual Braille dots to a standard height. The dot separation on those printers ranges from ten dots per inch to perhaps 17 dots per inch. Compare this with the typical 96 dots per inch of an image on a computer monitor and you will understand the issues regarding spatial sampling mentioned above. When the image is re-sampled down to a level that is consistent with the number of dots in an embossed image, much of the detail simply disappears.

Newer Braille printers have dot resolutions of up to 25 dots per inch, which is still very low in comparison with onscreen images. Some of them can also raise dots to variable heights to simulate gray scale imaging. Although there is no published information as to how many “gray levels” can be understood by an experienced blind user of an embosser with variable dot height. Prof. Baldwin estimates that such a user can probably recognize black, white, and perhaps three gray levels in between.

Typical embossing techniques either convert the 16 million colors in a bitmap image to black and white through the application of a single intensity threshold, or convert the 16 million colors to black, white, and several shades of gray through the application of several intensity thresholds. The result is that many colored pixels that are clearly distinguishable in the original image become indistinguishable in a four or five-level gray scale version of the image. Detail that depends on the recognition of different colors simply disappears. Therefore, many of the salient features of the image are often lost in the embossing process.

Prof. Baldwin has developed a mathematical image processing algorithm which, in many cases, preserves much more detail than the typical intensity-based gray scale approach. This algorithm converts the original image to either black and white or black and white plus three levels of gray, based not on absolute colors, but rather based on changes in color. The result is that images processed using this algorithm tend to have black outlines that define the salient features of the original image. In many cases, this produces more meaningful embossed images than the typical approach based on the direct conversion of color intensity to gray scale.

Prof. Baldwin’s solution to the spatial sampling issue is to make it possible in his programs to subdivide an enhanced image into panels, which can be individually embossed and then assembled into a poster-size tactile image. While not an ideal solution, short of developing an embossing method with an improved dot resolution, that is the best he has to offer.

Prof. Baldwin and Ms. Lacy have packaged the image enhancement algorithm in a free computer program designed specifically for use with embossers that support a single dot height and accept Duxbury graphics files in the .sig format as input. This program is named JpgToSig-A-01. It accepts any of several different bitmap image files as input and writes the enhanced version of the image into an output sig file. While it is possible for blind students to use the program, it is primarily targeted for use by teachers and others who assist blind students.

Prof. Baldwin has also packaged the algorithm in a free computer program named ShapeExtractor02 that is designed for use with any embossing method that can accept jpeg image files as input. This program accepts any of several different bitmap image files as input and writes the enhanced version of the image into an output jpeg file. Both of these programs can also be freely downloaded here along with the program named SvgDraw01. Note that all three of these programs require the Windows operating system.

These programs can be used with bitmap images from any source. However, in the world of education for blind students, the images that need to be embossed are often contained in electronic pdf versions of course textbooks. Prof. Baldwin uses the following procedure to assist Ms. Lacy in embossing images from her physics textbook.

1. Open the pdf file in the free version of Adobe Acrobat and locate the image of interest.
2. Use the zoom capability of Acrobat to make the image as large as possible while still fitting on the screen.
3. Hold down the shift key and press the Print Screen key. This saves the current screen image on the clipboard.
4. Open any of many available image editing programs such as the program named Lview Pro.
5. Paste the clipboard into the image editor.
6. Crop the image out of the surrounding material retaining only the material necessary to contain the image.
7. Save the cropped image.
8. Open either JpgToSig-A-01 or ShapeExtractor02 and follow the usage instructions to convert the image to the desired black and white or black, white, and gray format.
9. Save the enhanced image in an output file and emboss it using the embossing method of choice.

Ms. Lacy is now enrolled in her second physics course and plans to complete several more.

- As told by Prof Baldwin