Posted on by

How to Find Out the CIDR Notation for a Subnet, Given an IP and Netmask

In the process of working on some contributions to Salt (a fantastic new configuration management system), I needed to figure out how to find the CIDR notation for a subnet, given only an IP address and netmask. There is a python module called netaddr which is good for this sort of stuff, but the module is not a core module and using it would have required this module to be added to Salt’s dependency list. So, I needed to find a good way of calculating this information without the aid of a Python module. To my surprise, documentation on how to do this doesn’t appear to be that common on the web, so I thought I’d share it.

But first, a little bit of explanation for those that may need it. An IPv4 address contains 4 octets, each ranging from 0 to 255 (for example, 56.78.123.45). However, this notation is really just shorthand, designed be easier for humans to read. An IP address is really just a a 32-bit number, with bits 1-8 making up the first octet, 9-16 making up the second, etc. 255 in binary is 11111111, while 0 would be 00000000. The network mask (or netmask) uses the same notation as an IP address. When converted from that notation to binary, the number of zeros at the end will let you know the size of the network. A network mask (in binary) will be several 1s followed by (in most cases) several 0s. In the case of 255.255.255.0, there are 24 1s followed by 8 0s:

CIDR notation for a network would be the starting IP of the network followed by /NN, where NN is equal to 32 – the number of zeros. Since there are 8 zeros at the end, a netmask of 255.255.255.0 would be a /24 network. These are quite common. For the IP address 56.78.123.45, the CIDR notation for the network would be 56.78.123.0/24. /24 networks are easy because they always start at zero, but what if the netmask was 255.255.252.0? How would you find the start of that network?

The answer lies in bitwise operations, specifically the bitwise AND. A bitwise AND will take two binary numbers and compare the first bit of both numbers to each other, then the second bit of each number, etc., and will yield a 1 when both are 1, and 0 if either (or both) are not 1. Remember that a netmask will always be 1s up to a certain point. That point represents the start of the network. So executing a bitwise AND on the IP and netmask will give you 1s where there were 1s in the IP address, but only up to the point where the network began. The rest of the bits in the result will all be zeros.

As you can see, the result is 56.78.120.0. Because there are 10 zeros at the end of the netmask, this is a /22 (32 – 10) network, making the CIDR notation for this network 56.78.120.0/22.

Of course, it’s not too fun to do these conversions every time you want to get this information. There is a command-line tool called ipcalc which does this for you quite easily (The link was for a web version, but you can get a command-line version of the same tool in most Linux distributions).

Let’s make a programming exercise out of this, though. In python, the bitwise operator is a single ampersand.

So, the below python code will give you the CIDR notation:

And here’s the output:

Posted on by

Creative Comment Spam

This is not a popular blog. It’s rarely updated, and fits a pretty specific niche. So, I use an external comment manager called IntenseDebate. It holds all first-time comments for approval, and allows you to mark them as spam or permit them. Permitted commenters will not be held for review after their initial approval.

Over 99% of the comments I get are spam, and most of them are quite easy to detect. But spammers are getting a little more creative, and I thought I’d let others share in my amusement. Here’s a recent spam message:

It almost looks authentic. It sure sounds very Linux-y. But it has nothing to do with the actual post, and uses a couple of nonsense words that don’t even show up in a web search.

This next one actually seems like it might have something to do with the post I wrote (which has to do with auto-selecting a USB headset when it is plugged in):

However, once again there is a nonsense word, this time in the opening sentence. Also, it seems to reference Ubuntu’s 11.10 “Oneiric Ocelot” release, which came out in November of 2011, as if it were in the future. Not to mention the fact that I made mention in the blog post that I am an Arch Linux user, not an Ubuntu user.

This next post, however, quite literally made me laugh out loud:

Now that’s just lazy.

Posted on by

HOWTO: play radio stations’ online streams in your media player

I like to listen to streaming radio while I work. However, sometimes having a web browser open is too much of a temptation to just start browsing and ignore the work that needs to be done. So, I recently figured out how to play streaming radio stations in my media player of choice. This post will walk you through the process so that you can do the same.

You will need the following:

  • Firefox
  • Firebug (Firefox extension)
  • Flash plugin (needed for most stations’ stream players)
  • Any media player capable of playing media URLs

Before we start, understand that this walkthrough won’t work for every kind of streaming radio station. You won’t get it to work for things like Pandora, Last.FM, etc. This covers streams for local radio stations which allow you to listen through their website.

To start, open Firefox, go to the radio station’s website, and start up the radio stream. This is to make sure that you are on the correct page. Next, hit Ctrl+Shift+C to bring up the Firebug control panel, which will appear at the bottom of your browser window. Firebug has several different panels (Console, HTML, etc.), and we will be using the Net panel to grab the radio stream’s URL, but first we need to make sure that it is enabled. Click where it says “Net”, and if the panel is disabled, you’ll see a message saying so, with instructions on enabling it.

Now that the Net panel is enabled, refresh the page and you’ll start seeing items appear below. These are all of the HTML GET and POST requests, one of which will (hopefully) be the radio stream. NOTE: when reloading the page, sometimes Firebug will go back to the HTML panel. If it does this, just click on the Net panel to go back to it.

Let the radio stream load and let it play for a few seconds. The far right of the net panel will show the access times for the GETs and POSTs. Most will be represented in terms of milliseconds (ms) and will be split into different colors representing the time spent on DNS lookups, connecting, waiting for a response, etc. What we are looking for here is a GET which should have an all-grey bar, with the access time continuing to count up as the stream continues to run (see below, click for full-size image).

Right-click on that line and select Copy Location, and you will have the stream’s URL in your clipboard. You can then play the URL in your media player. For instance, in VLC this is done by clicking Media, then Open Network Stream.

Most stream URLs contain a bunch of querystring data, which in most cases (at least in my experience) is not needed to play the stream. You can try removing everything starting with the question mark and see if the stream still plays.

Since I prefer doing most things in a terminal instead of a GUI, my preference is to use mplayer to play the stream from the command line. To keep from having to remember the URL, I use an alias:

The -cache 128 parameter will make mplayer buffer the stream so that a temporary slowdown or loss of connection will be less likely to make the stream stop playing.

Posted on by

Back from the dead?

So, I’ve been away for awhile. A busy holiday season at work took away most of my November and December, and I did some traveling around the holidays.

But I haven’t just been slacking off! I’ve been teaching myself Python, and both tinkering with and contributing to a new(-ish) remote execution and configuration management framework called Salt.

Salt (http://saltstack.org/) is an amazingly simple-to-use tool which can be used to run commands on a number of hosts in parallel. In addition, it can be used to set and enforce configuration policies, filling the configuration managment niche also occupied by the likes of Puppet, Cfengine, Chef, Bcfg2, et al.

I’m currently evaluating Salt for potential deployment at work.

Also, after a number of enthusiastic recommendations, I decided to finally try WeeChat yesterday. I can’t believe I didn’t do this sooner! It has now replaced irssi as my IRC client of choice.

These days I’ve been hanging out regularly in #salt and #tmux on freenode (and occasionally in #mutt, #archlinux, and #python), with nickname archtaku. Feel free to look me up.

Posted on by

Setting a USB Headset as the Default PulseAudio Device Automatically Using Udev

USB audio peripherals still don’t “just work” in Linux, but they’re starting to get a little easier to use.

The last time I tried using PulseAudio, a little over two years ago, I experienced no end of frustration. This probably had as much to do with a lack of understanding as anything else, but regardless, I have been using nothing but ALSA since then. However, ALSA’s USB support is not very good. Changing the default audio device requires an application to be restarted to recognize the new default device. So, I figured I’d give PulseAudio another try and see if I could make my USB headset “just work” this time around. To my surprise, it was quite easy to do so, and in this post I will detail the steps I took. I did this on Arch Linux, but the steps should apply in most Linux distributions.

Before we start, we need to ensure that PulseAudio is A) installed, and B) running. Most distributions will take care of both of these for you, but due to the do-it-yourself nature of Arch Linux, in my case it needed to be installed:

Notice that I also installed two other packages, pulseaudio-alsa and pavucontrol. The former contains the ALSA plugin for PulseAudio, the latter is a helpful application for configuring PulseAudio and visualizing your PulseAudio configuration. Most Linux distributions will include the ALSA plugin (and may name the package differently), but you may need to manually install pavucontrol.

So, now that I have PulseAudio installed, it needs to be running. If you are using a desktop environment like KDE, Gnome, etc., PulseAudio should be automatically started when you login. To see if PulseAudio is running, check for it in the process table.

OK, PulseAudio is both installed and running. Open pavucontrol and click on the Output Devices tab. In the upper-right corner of each device will be a few buttons. The green one with a check mark on it (outlined in red in the example to the right, click for a full-size image) shows which device is the default output device. The Input Devices tab contains its own default as well, but for now we will focus on the output devices, for the sake of simplicity.

Start playing something, preferably in a PulseAudio-compatible application. For this example, I started a movie file in mplayer from the command-line, forcing PulseAudio output using the -ao (audio output) option:

With something now generating audio output, plug in the USB headset, and you’ll notice a new device show up in the Output Devices tab. But you’ll also notice that the audio isn’t going to the headset yet.

This can be fixed by clicking the green button next to the headset device, but this would have to be repeated every time the headset is plugged in, which would be an inconvenience to say the very least.

The solution to this is to write a udev rule which will detect when this device is plugged in, and then run a script to set the headset as the default input and output device. I’ll explain how to do this in a bit, but first we need to understand what it is that we want to do with this script.

PulseAudio has a command-line tool called pacmd which allows you to view and change the PulseAudio configuration. Plug in the USB headset, and then run pacmd dump to view the current configuration. There is quite a bit of output, so I have cut out all but the relevant parts:

PulseAudio calls output devices “sinks”, and input devices “sources”. We can see from this output that the default sink and source belong to the onboard audio card. To set the headset as the default device, we can use pacmd to set the default sink and source to the ones for the headset. This would be done with the following two commands:

Try runnng these commands yourself (replacing the sink/source names with the ones you observed when running pacmd dump), and you will notice that the headset is now the default input and output device if you check the pavucontrol window.

Now that we know how to set the headset as the default device via the command line, we can write a script that will do this for us. However, because this script will eventually be run as root by udev, we need to use the su command to run it as the user. The following script will check the process table and then, for each user who is running PulseAudio, will run the two commands needed to set the headset as the default input and output device.

Save this script to /usr/local/bin/usb-headset-set-default. It should be executable, and owned by root. Of course, you should also make sure to replace the sink and source in the script with the ones you observed when you ran pacmd dump, if they differ. Unplug the headset and plug it back in, then run the script as root, and in pavucontrol you should see that it has been set as the default input and output device. Note that there are a couple commands commented out in the script. These can be uncommented for debugging later on, if needed.

Next, we need a way to detect when the headset is plugged in. We can do this using udev.

Udev is what handles setting up device nodes for Linux. When you insert a USB flash drive, and it gets assigned a device name (like /dev/sdb), udev is what takes care of allocating this device name. Plugging in the headset will generate a number of device nodes. We can use udevadm to see each of them. Unplug the USB headset, then run the command below. After running it, plug in the USB headset and you should see some output appear.

Note: udevadm must be run as root.

Using another udevadm command, we can find the udev attributes for one of these device nodes. I have chosen /dev/audio1.

Wow, that’s a lot of output. Before proceeding to write a udev rule, we need to know how to utilize the attributes above to come up with a unique rule. This page contains an excellent walkthrough on how to write a udev rule, so go ahead and read it before continuing. The information about the udev commands is a little outdated (for instance, most of the commands referenced are now part of udevadm), but the information on rule syntax is very good.

So, we need to come up with a set of attributes that will match, but we also would like to find a set of attributes that are unique. Multiple matches will result in the script being run multiple times. In this case, where we’re just running a couple commands to set the default audio devices, there aren’t really any consequences to running the script multiple times, but it’s still a good idea to come up with a unique set of attributes. The rule I came up with was:

I saved this rule to /etc/udev/rules.d/85-usb-headset.rules. Notice that the RUN parameter is set to /usr/local/bin/run-script-in-background /usr/local/bin/usb-headset-set-default. In the process of debugging this rule, I noticed that the script was running, but the headset wasn’t being set as the default sink/source. This is where the commented-out debugging lines come in. With the debugging commands uncommented, I noticed in the log file that the headset wasn’t showing up in the pacmd dump output. It turns out that PulseAudio doesn’t know anything about the headset until udev completes setting up all the device nodes. This makes perfect sense. However, if udev has matched the rule and is running the script, this suspends further udev processing until the script completes. So, what will happen is that the script tries to set devices that don’t yet exist, and in effect nothing happens. To get around this, we need a way for the command executed by udev to exit quickly, but allow for a short delay so that udev can finish doing it’s thing before pacmd is invoked. So, I created the following script and saved it to /usr/local/bin/run-script-in-background:

This script simply takes the arguments fed to it and runs them as a separate process, in the background. This allows run-script-in-background to quickly exit so that udev can finish.

With the rule and scripts now in place, plug in the USB headset again and you should notice in pavucontrol that the headset is now the default device. If this is not the case, then use the debugging lines in the usb-headset-set-default script while tailing the log file to determine the problem. If the headset doesn’t show up in the pacmd dump output, then you might need to increase the sleep time at the beginning of the script.

I’ve been using this udev rule and script for several days now and the headset is working flawlessly. Hopefully, my instructions can help you get it working for yourself. All in all, I am very impressed with how well PulseAudio is working for me, given the nightmares I experienced in the past.

Posted on by

MiniDLNA: A Lightweight UPnP Media Server

For several years, I have been using MediaTomb as a streaming media server. Functionally, MediaTomb works fantastically, but a few things about it have bugged me for some time:

  1. The arrangement of media files. MediaTomb creates two hierarchies of file listings, called PC Directory and Folders. The former lets you browse from the root of your filesystem, and is a bit useless when the directory you are sharing is several levels down. Browsing through those levels each time you want to watch something is annoying. So I prefer the Folders listing, which places your shared folders at the top level and lets you drill down to subdirectories from there. Here’s where we get to the problem: While timed folder rescans and inotify allow you to add new files to the media browser simply by copying/moving those files to a watched directory, new files added in this fashion will show up in their own top-level directory under the Folders view, rather than staying organized. The only way around this is to use the web interface to delete the watched folder and re-add it, forcing a full re-scan of the media directory. It’s hard to tell whether this is a MediaTomb issue or a problem with the PS3′s DLNA client.
  2. Configuration Syntax. The configuration file for MediaTomb is in XML format. I have never been a fan of XML syntax in a config file.
  3. Streaming to a PlayStation 3 is not supported “out-of-box”. It’s simple enough to enable (in fact the option is present in the default config file, commented out), but this is an annoyance nonetheless.
  4. MediaTomb is not actively maintained. Releases are few and far between, the last of them coming over a year and a half ago.

That is not to say that that MediaTomb is without redeeming features. For one, it allows you to write scripts to transcode media types which it cannot handle. You won’t be able to seek or pause when viewing a transcoded file, but this is actually not a MediaTomb limitation. The very nature of transcoded media is that it is being converted on-the-fly, so seeking just isn’t possible. Another nice feature of MediaTomb is that its folder layout is scriptable, allowing you to control how the media files are presented in the file browser. Perhaps this would allow me to come up with a solution to my file arrangement problem, but I just don’t have the time or inclination to try to fix it.

A few days ago, while browsing Reddit, I saw a recommendation to use MiniDLNA to stream media. It has turned out to be a very simple to use alternative, and I have now officially switched to it from MediaTomb.

MiniDLNA is written and maintained by a Netgear employee and is used in their ReadyNAS line of storage devices. It is both actively maintained and easy to use.

MiniDLNA is not as feature-rich as MediaTomb. It is not scriptable, and it cannot transcode media. But what you sacrifice in features, you gain in sheer ease of use. You need only set one configuration variable in /etc/mediatomb.conf to get it up and running:

Additional media_dir lines can be added to share other directories. Streaming to the PS3 is supported without any special configuration needing to be done.

And that’s it! There are, of course, other things you should set up. First, you can change the server name shown to clients by setting the friendly_name value. Also, while not required, enabling MiniDLNA’s cache (which will keep it from needing to re-scan everything when it is restarted), as well as logging, are both good ideas. You’ll need to set up directories for both, which are writable by the nobody user (the user under which MiniDLNA runs). This can be accomplished by running the following two commands as root:

Then configure the following two parameters in the config file:

MiniDLNA has proven to be perfect for me. I don’t do any transcoding or scripting with MediaTomb, so I am not missing those features. MiniDLNA supports inotify to add new files to the media browser automatically, and does a better job of organizing files added to the watched folders. They stay organized in the the proper folder, but sometimes new files appear at the bottom of that folder’s file list rather than in place alphabetically. However, this doesn’t seem to be a MiniDLNA bug, but rather a bug in the PS3′s DLNA client since the files appear in the correct order the next time the PS3 is started up, or after telling the PS3 to do a new search for media servers.

All in all, if you want a simple alternative to MediaTomb, then I highly recommend MiniDLNA.

Posted on by

Clipboard Access from the Command Line with xsel/xclip

If you’ve ever found yourself wanting to copy the entire contents of a text file to the clipboard, you may end up doing something like the following:

  1. Open the file in a text editor.
  2. Select all text.
  3. Copy to clipboard.

It can be a blow to your productivity to stop what you’re doing and do the above, especially if you were working at the command line. Another disadvantage to doing it this way is that, if you close the program from which you copied the text, it clears the clipboard. If you forget this (or you simply didn’t know about this little quirk), then you may end up wasting more time repeating those steps to get that text into your clipboard.

xsel and xclip are two commands that allow you to interact with the X clipboards. Before I explain how to use them though, a short overview of the X clipboards is in order.

X11 has not one, not two, but three clipboards. They are called:

  • PRIMARY – Also known as the “primary selection” or the “primary clipboard”. This clipboard is populated whenever you highlight text with the mouse. If you’ve ever highlighted text and noticed that you can paste it by clicking the middle button on your mouse, this is the clipboard being used.
  • SECONDARY – This clipboard is very rarely used anymore, but exists to provide a “secondary selection” clipboard to accompany the primary selection.
  • CLIPBOARD – This is the clipboard you are likely most familiar with. It is the one used when you copy text from an application such as a web browser, or a GUI text editor like gedit.

For our demonstration, we’ll use the CLIPBOARD selection as it is the clipboard you’re most likely to be using on a regular basis. To begin, let’s create a file we can use to test:

You can verify the file has been created using the cat command to print it to standard output.

So, say that we’d like to get the contents of foo.txt to the clipboard. With xsel, you can do it like so:

The -i option tells xsel to read from standard input. In this case, standard input is being piped in from the cat command. The b option tells xsel to use the CLIPBOARD selection. To use the PRIMARY selection, you’d replace the b with p, and to use SECONDARY you’d replace it with s.

To copy the contents of the text file to the clipboard using xclip, you can use the following:

To use the PRIMARY selection, you’d replace clipboard with primary, and to use SECONDARY you’d replace it with secondary.

If you press Ctrl-V in another application, you’ll notice that the words “Hello world” were pasted.

In addition to copying standard input to the clipboards, xsel and xclip both have the ability to print the contents of the clipboards to standard output. Go ahead and select some text with your mouse and copy it to the clipboard using Ctrl-C, then try the commands below. (I will copy a few words from the previous sentence as an example.)

In both cases, -o is used to tell the command to print the contents of the specified clipboard to standard output.

The arguments to xclip can be abbreviated, so you don’t have to type -selection clipboard every time you want to use it. For example:

Both commands have additional features not described here, so check their man pages for more ideas.

Hopefully these commands can help you save time and increase your productivity when working from the command line. They’ve certainly done so for me.

Posted on by

The Reliability of X.Org

X11

Wednesday’s XKCD was funny as usual, but there was also quite a bit of truth to it. Since X.Org was forked from XFree86 several years back, I can count on one hand the number of times I have had to edit my X11 configuration, and most (if not all) of them have involved AMD/ATI’s Catalyst driver. This is a far cry from my early days experimenting with Slackware 7, when I knew little about Linux and even less about X11. Were it not for the helpfulness of more experienced Linux users on message boards and mailing lists (not to mention my stubbornness and natural curiosity), I may well have given up on learning Linux. In fact, aside from when I used to use the Catalyst driver, I haven’t even had an xorg.conf in the last several years. Gone are the days when I had to manually configure my display modes, enter the right parameters to enable mouse wheel scrolling, etc. I have been so spoiled by X.Org that, until I saw this comic on Wednesday, I had forgotten just how cumbersome X11 once was.

This is not to say that X.Org is all sunshine and roses. Features like multiple monitors and two-finger scrolling for trackpads don’t always work well, even with desktop environments stepping up in the last couple years to make configuration easier.

All in all though, when compared to the state of X11 about 10 years ago, X.Org has proven to be a fantastic implementation that has given Linux users little to complain about.

Posted on by

Printing to PDF in Mutt

When using my netbook, I don’t typically have a printer available. My printer is connected to my wife’s desktop computer which doesn’t stay on all the time, so I can’t rely on sharing the printer to the network and sending documents to it through her computer. Moreover, I’m not all that fond of printing things when saving as a PDF will do.

My email client of choice is Mutt. By default, Mutt uses the lpr command to print, which sends the message to the default printer. But I don’t want to actually print the message, I’d like a PDF instead. Luckily for me, as with pretty much anything in Mutt, it is possible to change the default behavior and supply a different command to print messages. Below you can see a short shell script I wrote which will create a PDF from the message contents:

The body of the email will be passed to the script by Mutt as the first argument, or $1, which I store in the INPUT variable. Notice that I’m also defining a couple other variables at the top of the script. The first, PDIR, is a directory where the created PDF should be located*. Secondly, I’m defining a command which should be executed to open the PDF once it has been created. I prefer a lightweight PDF viewer called zathura, but any PDF viewer (evince, okular, etc.) can be substituted here.

Just below these variable definitions, I make sure that the commmand-line tools I’m using to create the PDF, (enscript and ps2pdf), are installed. There’s not much of a reason to try creating the PDF if they’re not present, so a helpful reminder printed to stderr will notify you in the event that they are not installed. In most Linux distributions, these tools are found in the enscript and ghostscript packages, respectively.

You may have noticed that I am not using the which command to determine if these commands are in my $PATH. A recent blog entry by Aaron Toponce alerted me to the fact that which is not consistently implemented across platforms, and may not set a proper exit status or write error messages to stderr. POSIX provides a command named “command” (what else?), which is consistent across platforms. Running it with -v will print the path of the first matching executable in your $PATH and set an exit status of 0. If there is no match, a non-zero (or false) exit status is set. I don’t care about the path to the executable, so this output is redirected to /dev/null.

If you would rather use a different font for the PDF, you can find a list of PostScript font names here.

When opening the PDF using the pre-defined command, all output is redirected to /dev/null. This is because GUI apps, when run from the command line, will typically print diagnostic messages, warnings, or errors to stdout/stderr. While these messages can be useful for troubleshooting purposes, any output from the script will be displayed in your Mutt window, which will look very ugly. So, this output is discarded.

In order to get Mutt to call the script in place of lpr, you will need to add the following line to your .muttrc file (use the path where you saved the script, of course):

Below is an example of how the result will look. If you do not like the layout, you can alter it by passing different parameters to the enscript command. Check the manpage and experiment to find what works best for you.

mutt_print

So, by replacing the print command with this script, instead of printing the message directly, your PDF viewer of choice will display the message for you. If you want to print it out from there, you can do so. I rarely have a need or desire to print, but this at least leaves me that option. If you want to keep the message, you’ll need to use your PDF viewer to save a copy of it, because it will be gone once you close the window. Again, choosing to remove the PDF is a personal preference of mine, so the last two lines of the script can be commented out or removed if you’d rather keep the PDFs.

* I could just place it under /tmp, but since emails tend to have sensitive or personally-identifiable information, I prefer to have the temp files located within my home directory, where I can manage directory permissions and keep these files private.