A friend of mine came to me with a problem last night. He was up against a deadline to send out a newsletter, and he only had one day to make it happen. The problem was, the addresses he planned to send the newsletter to were contained in more than 1700 individual e-mails. Extracting them one by one would take hours, and he just didn’t have the time. Surely there had to be a technical solution? After all, take a look around. Everyone has a computer in their pocket.
The thing is, the word “computer” has different meanings depending on your life experience. For example, when I heard the e-mails were contained in Microsoft Outlook, I groaned. Why? Because my decades of experience in these matters hath shewn that when it comes to doing something other than cutting off Netscape’s air supply, Microsoft isn’t really all that motivated. Meanwhile, their software is famous for fighting its users on the beaches and in the hills to prevent them from getting at their own data. That is if it isn’t already busy destroying that data.
Fortunately, the e-mails had been first collected by Gmail, which wouldn’t normally be much better, except that Google was practically forced not long ago to give users access to their own information so it can be exported elsewhere. This meant it was possible to download all 1700+ e-mails. So we ended up with a 7MB file crammed with raw e-mail data.
Remember my point about the word “computer” having different meanings depending on your experience? Well, I’m an old tankard-wielding grognard when it comes to computing. I do all my best work on Linux, which is a PC-compatible operating system largely inspiried by the UNIX system of the 1970s. UNIX was invented at a time when we used computers more for computing and less for posting our status on social media and browsing the web with an obsolete kludge of an application called a “browser” that for reasons passing understanding needs more than 1GB of RAM to operate correctly.
Now my friend wasn’t completely without options. He had a Visual Basic program that purported to do what he needed, but it wasn’t working properly due to errors in its variable declarations. This normally wouldn’t be a problem since I have voluminous experience with Visual Basic, except Visual Basic is a Microsoft product, and Microsoft software will not cooperate. It never has, it never will. In fact, it will fight to prevent you from getting your work done. Even if I had a way to load, debug and recompile a Visual Basic program in 2019 (oh dear), there’s no guarantee it would work. The term for this is “technical risk.” When you only have a few hours, you can’t rely on software that might work. You need software that will work.
So Linux it was.
I took a look at the e-mails. Each line with the address we needed to extract had the common label “E-mail address:” next to it. So I did the extraction job with one command. I used grep to copy all the lines from the big text file with that label in them. That gave me a file with all the addresses preceded by “E-mail address:” Then I opened the file in Emacs and replaced all the labels with a blank space. Now I had a file with a list of all the addresses. My friend needed them comma-delimited, which means all the addresses should be separated by commas instead of on their own line. So I used sed to remove the newlines and replace them with commas. The entire process took six minutes.
People often want to know “what’s the big deal about Linux? What can I do with Linux I can’t do with Windows?” Funny you should ask.
Why was it so simple on Linux and simultaneously close to impossible on Windows? It’s the same computer. It’s the same hardware. If the processor can do the work on one, it should be able to do the work on the other, right? Well, not so fast. Microsoft is in the business of making sure every PC has a copy of Windows glued to it and making sure every business on Earth is locked into Microsoft Office. They are not at all interested in solving problems with computers, so they don’t provide their users with the tools they need to get jobs like the last-minute newsletter done. Oh sure, you could try to do it with various Office applications, but it would be about the same experience as repairing a truck engine with your teeth. The first commercial versions of Windows were available more than 30 years ago. The system still doesn’t have even a mediocre text editor. The difference between Emacs and Microsoft’s Notepad are analogous to the difference between a bengal tiger and the bacteria under your refrigerator.
I will concede in advance that I have had my moments with Visual Basic and VBA. Visual Basic is one of the rare bright spots in the constellation of Microsoft products, along with DirectX and Windows 95. However, those few successes do not make up for the unholy siege of trying to get useful work done with their other products, or using the browser-that-shall-not-be-named.
But the newsletter project itself really isn’t the point. What I did yesterday afternoon wasn’t remarkable in any way when it comes to computing. Running the output of grep through a couple of pipes to remove newlines and comma-delimit a list of e-mail addresses is pretty basic stuff if you have the experience and you’ve been trained to look at problems in the Linux way. We’ve been doing things the Linux way for more than 25 years now. If you have an Android phone, so have you.
What I ultimately accomplished yesterday was to save a man a few hours of tedious error-prone hard work and deliver something important on a deadline. I used powerful tools to automate a task. I was able to do this because I’ve put a lot of years into learning how to use those powerful tools. My friend knows this, which is why my phone rang yesterday. When you need an engine, you call an engineer.
But according to America’s employers and their robots, I have no marketable skills.
Automating things has been the theme of my career, both as a contract programmer and in my own businesses. I wrote software that takes a plain text book manuscript and a cover image and compiles them into a validated EPUB3 electronic book that can be published on nearly any retail bookstore. Hand-coding an EPUB3 would take days, even for someone familiar with the process. You really have no idea how many apostrophes there are in a science fiction novel until you have to replace them all with HTML entities. Excel certainly isn’t going to help you. Being able to convert a manuscript to a book in a few seconds is one of the key reasons I’m able to publish so quickly and have a successful career as an author. Turns out readers appreciate a book that isn’t riddled with errors.
When I worked for a major firmware manufacturer, my team authored in-house software that automatically wrote makefiles for commercial BIOS images to install in retail PCs and laptops. The BIOS, or Basic Input-Output System used to be the first software that ran when you turn on a PC. The makefile was the list of instructions for an application called a “compiler” to build each BIOS image so it could be installed. Makefiles for such software could easily run to hundreds of lines. If there is a single mistake or typo, an hour-long build might fail. Multiply that by the hundreds and hundreds of test builds we were required to do for companies like Honeywell, Toshiba and Compaq, and a mistake could easily cost the company hundreds of thousands of dollars, or millions if a faulty BIOS found its way into a production PC. Once we were done automating it, however, the errors were gone. The software we wrote perpetually saved the company millions of dollars a month.
But, according to America’s employers and their robots, I have no marketable skills.
I wrote an optical document processing system for a financial services company using, ironically enough, Visual Basic. Our task was to create a printer driver for each of the unique forms a car dealer might use to sell or lease a vehicle. The printer had to write name, address, mileage, etc. in various boxes on each form, and the printer driver’s job was to tell it where to move the print head so the information was actually inside the pre-printed box. When I arrived at this company, they were hand-coding the printer drivers with a ruler, a pencil and a calculator. When I left, they could scan a document, arrange the data fields visually on the screen using each unique form as a backdrop, and have my system automatically write the driver. Productivity went up more than 2000%. It took me three weeks to develop that system.
But according to America’s employers and their robots, I have no marketable skills.
My first job out of college was as a portfolio analyst at a boutique brokerage firm in Orange County, California. Our department’s job was to analyze a potential client’s investments and generate a snapshot based on the current prices of their holdings. When I arrived at this company, one of my colleagues was seated in front of Microsoft Excel with a calculator, adding up stock shares and prices. When I left, the VBA code I wrote in Microsoft Excel would automatically dial into a bulletin board service, download the prices and assign them to the appropriate holdings. This time, productivity went up 6000%. We took a backlog of 40 portfolios and knocked them out in an afternoon. The fees our brokerage earned with that system were staggering, to say the least, considering the impressive lists of assets our clients brought us. They easily took in an additional seven figures a year with my system. Probably more. Took me about a month to write and test the code. Plus the cost of a Compuserve subscription.
But according to America’s employers and their robots, I have no marketable skills.
Let’s be fair, and go back to the newsletter project. I saved a man a couple hours of work. Big deal. We saved the cost of three large pizzas. Fair enough.
What if I save 1000 people a couple hours of work? That’s a lot of pizza. How about 50000 people? What if I wrote software that saved 50000 moderately well-paid office workers four hours of work a month? Say each of those workers makes $20 an hour. That’s four million dollars a month. Can I do it? Well, I’ve been automating things my entire career, right up to yesterday afternoon. Along the way, my employers and clients have saved millions upon uncounted millions avoiding errors and even more millions selling investments with the documents and electronic components my software wrote for them. In the words of Montgomery Scott, “would that be worth something to ye?”
Apparently not, because according to America’s employers and their robots, I have no marketable skills.