Learning Perl adds DOS command examples

Windows commands get short shrift when we write about system, and I’m just as guilty as anyone else. In the “Process Management” chapter Learning Perl, we barely mention Windows. In the seventh edition, I want to change that to have a Windows example for each Unix example if I can develop a reasonable homologue. With Stackoverflow, that’s not so hard.

We started the chapter with the “Unix date command”:

system 'date';

This one is almost easy because DOS also has a command named date, but by itself it prompts for a new date. With the /T date it doesn’t prompt:

system 'date /T';

The next one isn’t so bad to translate either. We used the ls command to demonstrate the use of a shell variable that Perl should not interpolate:

system 'ls -l $HOME';

We know that’s harder than using ~ to specify the home directory but that’s not the point of the example. The homologous Windows command does not have the interpolation problem:

system 'dir %userprofile%';

The Stackoverflow answer to What is the alternative for ~ (user’s home directory) on Windows command prompt? has some other options, but this one seems the least complicated.

Putting an application in the background is easy on a Unix shell:

system 'some_command &';

In Windows, there’s the start command. It doesn’t put the process in the background but it separates it so the parent command can continue:

system 'start /B somecommand';

The C<< /B >> suppresses the creation of a new window. There are some issues about which environment the new command gets, but it can go off to do its work while the main program continues. Stackoverflow has an an answer for Powershell, but I’m going to skip Powershell in Learning Perl. I’ll have to explore that in a different post.

The M+ console font

Sinan Ünür sent me a screenshot of his use of the console font M+. He uses ConEmu, one of the tabbed console programs I showed in Windows Command Shells. It has the same failings that they all do since they all use the same underpinnings, but it’s other features are nice. He’s talked about some of his struggles

Sinan needed to show exponents (or superscripts), so he needed a font that had the correct glyphs for that. That’s one of the huge problems for these consoles since the font support if very rudimentary. With that, font suppliers aren’t motivated to support much of the Universal Character Set.

After installing the font, configuring ConEmu to use it, and setting the code page to 65001 (chcp 65001), he gets the superscripts that he wants.

The Command Prompt and Fonts

In Windows Command Shells, I wrote about about the Command Prompt’s inability to handle Unicode. I noted that this problem was really with the infrastructure behind the Command Prompt, so other console programs that use the same infrastructure have the same problem.

That’s not really true. If I send UTF-8 octets to the terminal, they get there intact. I can redirect the output to a file then look at it in an appropriate editor to see the correct data.

The problem is that the terminal can’t display the characters, for reasons that Raymond Chen writes about in Why are console windows limited to Lucida Console and raster fonts?. The right data are there, but the fonts to display them don’t have the glyphs.

The Command Prompt doesn’t have a way to add fonts to it’s list of available fonts. No matter how many fonts you install in Windows, that list in Properties will be the same.

However, you can use regedit to change that list. You have to change or add a string value under \HKEY_LOCAL_MACHINE
SOFTWARE\Microsoft\Windows NT\CurrentVersion\Console\TrueTypeFont

The name has to be a series of zeroes. I don’t know why. The font name has to be the name shown in the \HKEY_LOCAL_MACHINE
SOFTWARE\Microsoft\Windows NT\CurrentVersion\Fonts
. The White Rabbit font works.

Now the console properties shows the new font.

I haven’t found a way to tell which fonts will work before going through all of those steps. Microsoft has four requirements for a TrueType font for the console.

  • Not italic
  • Can’t have negative A or C space, which means it can’t stick out of it’s box to either the left or right. The A and C space is the padding inside the box, and negative space means there’s some overflow.
  • Must be FF_MODERN if it’s TrueType. I don’t know which bit of the TrueType properties this looks at. I think you can write a C++ program using the Windows API to check this.
  • Must be OEM_CHARSET if it’s not TrueType. I don’t know how you get this from a font.

The White Rabbit font meets all of these requirements, but that doesn’t matter. It still doesn’t have the glyphs I’d want. That leaves me where I started. Almost all consoles on Windows are inadequate until someone makes a font with the right properties.

Further reading

Looking for Win32::OLE examples

I’m looking for some cool Win32::OLE examples for controlling Windows apps from Perl. Reply, email me, post a gist, or whatever.

Leo Lapworth’s video tutorials

Leo Lapworth made some simple video tutorials on installing Perl and writing your first program on Windows.

There are many other video tutorials on Youtube, but most of them are just telling you how to navigate a web page. I can sum them up as “ActiveState“.

Areas where Windows Perl has problems

There are some areas where Perl on Windows doesn’t quite work. In some of these, such as forking, that’s because Windows doesn’t work like that. In others, it’s because the code people have created with Perl, isn’t portable.

I’m collecting some of those problems here and will improve on this post as I get feedback. This is more of a scratchpad for future thing than a fleshed-out idea. You can help!

Forking and asynchronous frameworks that fork

Mod_perl, a plugin to embed Perl inside apache, used to be a problem on Windows because apache used to be a problem on Windows. Apache 1.x wanted to fork. Apache 2.x targeted Windows with another concurrent processing module to fix that. perl.apache.org has instructions for Apache 2.x and Perl.

Perl has several asynchronous frameworks, but if they are based on fork, they will have trouble on Windows. If your web framework depends on one of those, it may not work either.

Some things to investigate: Parallel::Prefork, IO::Ascync, EV, Thrall. Does anyone want to do the shootout for those?

Python on Windows is hard too

I’ve been looking around at other languages to see how they adapt to the Windows environment. Sinan Ünür sent me a link to Jessica McKellar’s “The Future of Python” from the New Zealand Python User Group. Get past the language bashing at the beginning (and see Tim Bunce’s TIOBE or not TIOBE – “Lies, damned lies, and statistics”), she has some interesting things to say about Python on Windows and how hard it is for newbies on Windows to get started.

If you continue past her Windows points, you can listen to her talk about games and that people “just want to make games”. But, that’s like saying I just want to build a bicycle when I really mean assemble parts I don’t really understand. There’s a lot I could write here about using Python or Perl to create a game development kit for “normal people”, but that’s not the point. The people making those kits still understand what’s happening, and those are the people these languages target.

I think this is why development outside of Windows specific technologies is so hard. We’re not meant to understand all that stuff behind the scenes, or to set our environment ourselves, or to muck around in the weeds. That’s why the big IDEs and fancy tools exist and expose just what we should be thinking about.

It’s not that any of this is wrong; we’re trying to do things outside of the scope of the environment.

Perl, Python, and many other languages work (enough) on Windows, but there’s an impedance mismatch in how those languages think about problems and want to do their work.

I’m still amazed that it works at all, and reminds me of Louis CK’s bit about the miracle of flight that is somehow is ruined for some people when the inflight wifi goes out.

Getting with the program

I’m not a Windows person, and I’m not trying to hide that. Part of this challenge I’ve given myself involves my discomfort and ignorance of the operating system I see on many a businessperson’s desk and often foisted onto their developer’s desks.

The pain of this process is apparent to anyone who didn’t start on Windows. They come into the situation trying to replicate the environment that they use on the non-Windows system. It’s not just a different operating system; it’s a different way of thinking.

I’m used to the Unix model where there are lots of small programs that cooperate with each other. I edit files in one program by run them in another. The unix pipeline model makes it easy to pass text around, and it’s in the DNA of those developers to write small tools based on standard input and output. When I first started this project, I thought I’d be able to get close to that.

I couldn’t get anywhere near to close. Whereas I’d use a terminal in unix land, the various Windows consoles have historical and severe problems handling modern text data (see Dealing with code pages). My first reaction is that Microsoft is stupid, but that’s not true. There are lots of smart people there, and the more I read from Raymond Chen, the more I appreciate what’s going on there and why things are the way they are. The consoles suck and are stuck in the 1980s because Windows people typically only use them for legacy stuff.

When in Rome, and all that. As I investigated these issues, I was in the land of not-Perl. All sorts of developers have the same problems, and the recurring answers all seem to point to the same thing: Windows development is about silos and single applications. It’s no accident that applications can go full screen and applications display all there stuff in one meta-window. They expect you to work there and stay there. I don’t like that because I beg the question that it’s the wrong way. It’s a different way, certainly, but it’s also the same way as Smalltalk, something I like.

In my Perl classes, I’ve mostly told Windows people to use Notepad and Command Prompt, but not primarily because I think they should develop like that. I encourage people to work in class the same way they plan to do their real work. I’ll adjust. But, training computers are often locked down. Perl is there often because I’ve gone through a process with an IT manager to get it installed. That’s typically a painful process with lots of back-and-forth emails. Getting them to license something like Komodo IDE is orders of magnitude for complicated. I know that Notepad and Command Prompt will be there, and I know that at the Learning Perl level the deficiencies in those tools don’t matter.

This explains how Komodo, Padre, and other things work. To see proper output for UTF-8 that I couldn’t coerce out of the Windows console backend, I resorted to Komodo Edit:

The IDE can do anything it likes because it can build from the ground up. Everything it needs it provides, so it doesn’t depend so tightly on history and internals.

This is what I need to get used to, I think, or at least tolerate for the length of this project. Komodo puts the command output under the program text pain without a way to have it anywhere else (optimally, a separate window). If I’m going to stay inside the IDE, how can I get the most bang for my buck with these huge monitors I have?

What’s your Windows Perl environment?

I’ve listed some tools in Text Editors for Windows Perl and Windows command shells. What do you use?

I know that many people use cygwin, and if you do feel free to tell me about it. I’m interested in how you edit files and how you run programs. If you feel like taking a screenshot of your Desktop showing your set up, I’m curious to see what colors you use too. What do you do to personalize your environment?

I’ve outlined my current environment, but that’s also changing as I try new things.

My Windows Environment

I’m not running a dedicated Windows box; I’ve installed (legal) copies of Windows in various releases in VMWare Fusion on my phat Mac Pro. I set up a big RAM disk from which to run everything, and have all the third-party stuff installed on a disk that I mount in the virtual machine. For the most part, I’m working in Windows 7.

I didn’t choose VMWare for any particular reason. I had it, but Virtual Box and several other options might have worked too. It hasn’t caused me any problems and it’s a lot faster than the dedicated hardware that’s sitting unplugged under my desk. But then, I am storing everything directly on RAM. The virtual environment is important so I can easily start from scratch and retry any instructions I develop without missing something because I forgot to remove a dependency.

For work, I use Sublime Text (although I’ve listed most of the other editors). I’m not doing anything that fancy, and I also do quite a bit of editing on my Mac with my normal development process. There’s good integration between both in Fusion so it works out nicely. I only feel slightly guilty about that since I’m not teaching Windows, just Perl on Windows.

For Perl, I have many installed. Most of the things I’m working on so far are standard Perl and don’t need anything fancy. Once you can install modules, the rest is mostly just regular Perl.