Geoffrey Swift's βlog

In an old post I detailed how to avoid advert videos on the 4oD service. I have again I made use of the redirect_program feature of the Squid web proxy, but this time the adverts from Channel Five's Demand Five service are being removed.

I set about analysing how Demand Five works, by using the Fiddler web debugger from Microsoft. It became apparent that similar to the Channel Four service, there was a play list of (Flash) video files in an XML format.

So I attempted using the exact same technique as for 4oD. I created a "man in the middle" to filter out the adverts from the Demand Five play list, using my own PHP and XSLT code. Unfortunately the doctored play list I generated was rejected by the Flash player as invalid, presumably because required elements (adverts) were lacking. It's not clear whether this an intentional feature in Demand Five or not! Either way a different technique was needed.

I have avoided seeing banner adverts for some time, by having Squid serve upa 1 pixel by 1 pixel transparent GIF instead. Since modifying the XML play list on the fly didn't work out, I decided to get Squid to serve up a video of 0 seconds duration in place of an advertisement.

I searched the web and found a suitable dummy Flash video of 0 seconds length to use. So I just needed to let Squid know I wanted the dummy video instead of the advert videos. To do this I added the following line was added to my existing Perl script squid-redirector.pl:

Now I can enjoy Neighbours without having to wait for the adverts to finish. This is particularly useful when the playback gets interrupted and you have to start from the beginning.

It's been a while since the last rant, here are a few serious concerns for web developers to consider!

Resending POST data

The "back" button is very useful feature to have in web browsers. Unfortunately you can be prevented from going back to a page that is generated in response to filling in a form.

When you try to do this, the browser explains that the page can't be displayed without resubmitting the form data. This is since there are special rules which generally prevent an HTTP POST response from being cached.

So you can either resubmit the form, but you might not get the same results the next time around. There is no option of just seeing the original version of the page. The browser could ask you whether you wanted to see the old version, and could perhaps warn you that showing you the old version would be in defiance of web standards. This has to be an improvement over what is essentially a flat refusal to behave intuitively.

Web programmers can work around this problem, by making a web server respond to a web browser's posted form data appropriately. The solution is to instruct the web browser to be redirected to another web page, which can retrieved conventionally. This page you're redirected to appears in the browser's history instead, so your back button can work as normal.

For further technical details, I recommend reading the HTTP RFCs and the Wikipedia article on the "Post Redirect Get" pattern.

Click to close window

Many web pages, particularly those in popup windows have a hypertext link entitled "Click to close window". I feel this is redundant, since web browsers windows can be closed like any other. Given that web browsers open new "windows" as tabs, the wording itself isn't quite right either.

The only case for having such a link is for popup windows created in such a way to specifically hide the standard close window option. Even so this appears to be a case of trying invent a rounder wheel.

Print friendly version

When viewing websites like Google maps, it can be useful to make a hardcopy so you can review the information later. Unfortunately what you see on screen doesn't necessarily work so well on paper. This is what prompts web developers to create a "printer friendly" version.

While this seems like a great idea, I feel this is really just a sorry excuse that they couldn't figure out how to make a web page that appears correctly both on screen and on paper.

There exists a useful feature in CSS2, which lets you define formatting based on the "media" being used. You can use this to make your web page to look totally different when printed.

For an example of this in action, try using the "print preview" option in your browser when looking at my website. You should see immediately that the navigation menu disappears and background colours are turned off.

A useful article on this can be found on about.com

The strace utility is well recommended for investigating all manner of Linux problems. But while using strace to debug a crash in the Linux version of my Talker application, I came across something which proved to be a red herring. It appeared that select() was failing and setting errno to ERESTARTNOHAND.

Without being side tracked by what that would have meant, the actual problem itself was actually a segmentation fault had happened in another thread. select() was actually setting errno = EINTR, due to the corresponding SIGSEGV signal.

The error outside of the main thread was only apparent when using strace -f. The "follow forks" option is necessary, since pthreads works by forking off new Light Weight Processes (LWPs).

I found plenty of posts on the web detailing similar issues, although no solutions offered. Hopefully this information will be helpful to someone else.

It's been quite a while since I last posted up a mix, although I have been getting a bit of practice in nonetheless. So here's a new drum and bass mix, to celebrate British Summer Time!

The tracks selected are all several years old, but don't expect to hear many 'classics'. This selection represents a selection of my favourite tracks which seemed to be off the radar of top DJs.

Will hopefully appeal to fans of the darker side of drum and bass. Feel free to comment…

The CHANGES file in Tinyfugue 5 explains that the user interface has changed: New screen handling. See "/help windows". What this means is that when changing between worlds, the whole screen is replaced to show only output from the selected world. Previously messages from each world would all appear in succession on screen.

As I multi-spod and hold conversations across several talkers at the same time, this new interface is not my personal preference. I would rather be using the time honoured classic, which is Tinyfugue 4. However I discovered recently that this older version no longer compiles on modern Linux distributions.

I had a look into this problem, and fixed the source code by retro fitting the relevant fixes already done in Tinyfugue 5. The changes were minimal, so it didn't take long to get this compiling for GCC 4 on Linux. I made a unified diff of these changes. This has come in handy twice for me now, so I hope this proves useful to someone else.

Implementing my radio player took more experimentation and research than actual coding. As with most software, things turn out more complicated than first anticipated!

To tune in into an Internet radio station, normally you would click on a link on the station's website. From this link your browser will download a play list file, which in turn contains the URL that your favourite media player can use to access the station's live stream. My goal was to make this accessible in a more convenient manner, by allowing access to several radio stations in one place.

So I decided to create my own play list file that contained all the relevant URLs for my favourite stations. I wrote a PHP script that would generate this this play list by downloading each station's play list and extracting the relevant stream URL.

Play list file formats

There are three main formats used for these play list files, which are ASX (Windows Media Player), M3U and PLS (WinAmp).

Easiest to parse was M3U, this is just an ordinary text file that has the URL as a line of text. As PLS is an INI file, this seemed easy using PHP's built in functions to parse INI files. I quickly found that the PHP functions were unreliable so I wrote a kludge to get the URL myself, using a simple regular expression.

In a previous article I discussed problems with ASX files not being XML, so I avoided using this format unless the station did not also offer M3U or PLS. By sheer luck the few stations I am accessing via ASX are well formed XML. Unfortunately the structure of these ASX files vary, so it took a few iterations of the XSLT code to extract the stream URL realiably.

The file format I ultimately create my play list is is XSPF, as this is a well defined format based upon XML. From the master XSPF play list, I can readily generate ASX, M3U and PLS by using XSLT. Having the playlists available for means good support for tuning in with various different media players. I would personally recommend using the XSPF play list in VLC Media Player.

Streaming from a web page

I felt the stations should ideally be available directly a browser, without requiring an external application to be installed. After trying a few different options I opted to use the "XSPF Web Music Player" written in Flash. Getting Flash working is an exercise in itself made easy using the UFO JavaScript library. This setup offers good compatibility with typical browser setups.

Fixing the stream URL

The biggest problem with this was presented by a strange behaviour of Shoutcast servers. When you access the stream URL for a Shoutcast server using a web browser, you are served up a web page about the stream. This HTML content is provided instead of the MP3 stream, on the basis of the User-Agent header of the HTTP request.

Generally browser plugins have their HTTP requests made for them by the browser, so HTML content is delivered to a Flash movie or a Quicktime regardless. The exception to this is the Windows Media Player plugin, which bypasses the browser completely; disregarding proxy settings etc.

Shoutcast servers do provide a special URL that allows you to ensure the MP3 stream is returned regardless of the User-Agent. You just append ;stream.nsv for example http://example.com:1234/ becomes http://example.com:1234/;stream.nsv.

Appending this string confuses any servers that aren't using Shoutcast though. So it became necessary to probe each server, to adjust the URL where required. This works by impersonating Firefox and when HTML comes back, that's Shoutcast.

Memory usage

One significant problem appears to be that of memory consumption while listening to a stream. If you're listening for a few hours or so, expect your memory usage to be in the 100s of megabytes. This is enough to cause problems in a typical PC setup. A workaround for this is to stop and restart the stream occasionally.

The reason behind this is that the MP3 stream is treated as if a very long MP3 file is being downloaded. What you've heard so far remains in memory even though, with the exception of Real Player, you can't seek backwards. This is a well known problem that affects nearly all media players and browser/plugin combinations except VLC Media Player.

Recommended use

If you use the M3U or PLS play lists, you may well just see IP addresses and port numbers rather than meaningful station names. This is a limitation of the file formats used, and isn't fixable. Using the XSPF play list is recommended as it has the station name and website link. Also you can use the XSPF play list in VLC Media Player, without having to worry about your memory usage.

I have been working with various playlist file formats as part of my internet radio project. This has involved creating XSPF playlists from XML sources and using XSLT to convert from XSPF to the alternative PLS and M3U formats.

According to the Simple ASX article on MSDN: an ASX file is an eXtensible Markup Language (XML)-based text file which references a Uniform Resource Locator (URL) for a piece of media content. Having read this I felt that ASX files ought fit neatly into my XML and XSLT based architecture. Only when implementing this, did I discover that ASX actually has quite limited compatibility with XML.

The XML validation tool xmllint rejected a couple of ASX files, since they included the copyright symbol © encoded using ISO-8859-1. It seemed easy to solved this problem, by explicitly specifying the encoding in the XML declaration e.g. <?xml version="1.0" encoding="iso-8859-1"?>. Although the file could then be validated as XML using xmllint, Windows Media Player would simply not load this modified ASX file. It was then apparent that ASX is not compatible with XML.

Some URLs referenced in ASX files may well have query strings, which are of course delimited with ampersands. After seeing a few of these I noticed that none of the ampersand characters were escaped as they should be in XML. For a well formed XML document, a literal ampersand charater should be represented using an escape sequence, e.g. &.

I edit an ASX file to make it valid XML, by ensuring the appropriate escape sequences were used. The play list came up in Windows Media Player, but it did not play as expected. When looking at the properties of the relevant track in Windows Media Player, the text & could be clearly seen as part of the URL. Although I had made an ASX file into a valid XML document, once again this was futile since the ASX file was not interpreted as intended.

To summarise here are some quirks of ASX files that make it incompatible with XML:

• The default encoding for ASX appears to be ISO-8859-1.
• An XML declaration cannot be included in an ASX file.
• Ampersand characters in ASX files are parsed as literal text and should not be escaped as &.

To work around these problems, I wrote a PHP script that downloads an ASX file and converts it in an XML format. To achieve this it simply escapes all the ampersands, and serves this modified content with an appropriate HTTP header for ISO-8859-1 encoding: Content-Type: text/xml; charset=iso-8859-1. Aside from XML incompatibilities many ASX files are tag soup. My PHP script therefore includes kludges to address problems specific to the ASX files I'm working with.

This wasn't quite the clean solution I originally had in mind, but I am now able to import from multiple ASX files across the web. It was less trouble working around PHP's woeful ini file parsing routines to get the PLS file format supported, but that's another story!

Having made my own list of favourite internet radio stations available online, I was quite interested to hear about the "Pirate Radio Toolbar." I downloaded and installed the toolbar, and found it offered an impressive list of pirate radio stations. Most are pirate radio stations that also stream over the internet, a lot of the others were internet based stations that also play underground music.

The toolbar software itself is created (or rather configured) using the Conduit website. Conduit allows anyone to sign up to create similar toolbars for commercial use. These toolbars can be configured to have many different features, one of which is a radio playlist.

My sole interest was the list of stations, since I regularly enjoy listening to internet radio. The rest of the tool bar offered nothing I wanted; it merely obscured the top of my web browser. It seemed much more appropriate to listen using a more fully featured purpose built program like VLC Media Player to tune in. With music playing in another program you can obviously hide the player behind your browser!

So I mused with the idea of extracting the relevant data into an appropriate format. Using the Fiddler HTTP debugger, the URL for their play list data became apparent quite quickly.

The play list for the Pirate Radio Toolbar (and other Conduit toolbars) arrives over the web in an XML format, which is ideal for manipulation with the XSLT (the programming language used for my website). So I wrote some XSLT code to convert the playlist into an XSPF format, ideal for use in VLC Media Player.

I wrote a shell script to download the Pirate Radio Toolbar station list using wget, and convert it using xsltproc and my XSLT stylesheet code. This script runs nightly using cron, so my copy of the station list should keep up to date with any new stations they add.

I even wrote a couple of extra XSLT stylesheets to convert XSPF to the PLS and extended M3u file formats. I make use of these stylesheets to generate my own dynamic radio playlist now. I might post this XSLT code up for download in case anyone else wants to do similar conversions.

Please visit my radio page for more details. I host my own radio station play list, which exclusively comprises high bitrate drum and bass, jungle and dubstep.

I am pleased to say that there is now a valid RSS 2.0 feed available for my blog. Previously there was just the Atom 1.0 feed, since this is format I am using for storage.

I couldn't find any available Atom to RSS conversion utilities, except for atom2rss.xsl from http://atom.geekhood.net/. I had tried using this a while ago, but found it would not produce valid RSS and even corrupted parts of my text through inappropriate handling of escape sequences.

Feeling a bit more confident in my XSLT, I set about addressing these issues. The feed validation service from w3.org now tells me that "this is a valid RSS feed", and so consider this to be good enough for my purposes.

It is slightly disappointing that RSS limits you to a single enclosure (podcast) for each item in a channel, whereas Atom allows for multiple enclosures. Because of this it is not recommended to use the RSS feed since you will actually lose out on some of the mp3 content.

My Atom feed has brought to light some cases that cause a problem with the XSLT code, and there are probably other valid Atom feeds that would not be translated to a valid RSS feed. Nonetheless I feel this is an improvement and am releasing this updated version into the public domain.

I've emailed my fixes over to porneL who originally wrote atom2rss.xsl, and he's kindly updated his site to include my changes.

I'm pleased to announce a brand new version of my radio tuner. This is now embedded into my website using a the XSPF Web Music Player, which is written in Flash. I hope other people will enjoy using it as much as I am, any feedback is welcome.

Here are a few oldskool jungle tracks I've written several years ago. Seemed about time to put them up in a podcast, as they weren't very prominently linked to before. Have been meaning to do this for a while, and if time allows more could I find time I might dig some more out.

These were all written using 8-bit breakbeat samples I created on the Atari ST using Mastersound. These were chopped up into tiny pieces, then sequenced using Fast Tracker 2 on the PC. Now turned into MP3z using XMPlay and Lame for your listening pleasure! Enjoy.

Getting Internet Explorer 7 to work with the HTML version of my internet radio tuner was no walk in the park. Read further to learn how to avoid the same pitfalls.

The first problem was that using <object type="application/x-mplayer2" data= … > yields the error "Internet Explorer has blocked this site from using an ActiveX control in an unsafe manner. As a result, this page may not display correctly." Puzzlingly I found that there is no such warning when using <embed type="application/x-mplayer2" src= … > instead.

Using <embed> rather than <object> just for the sake of Internet Explorer didn't make me feel entirely happy, as this proprietary tag is not valid HTML. To solve this problem and retain the validity of my HTML, I chose to use the even more proprietary trick specific to Internet Explorer - conditional comments. This means that the IE specific <embed> is effectively commented out, and the <object> tag is ignored by IE. See some example HTML below:

This was slightly tricky to get right in the XSLT. I could have tried using xsl:comment and CDATA sections, but xsl:text and disable-output-escaping seemed like less trouble:

Problem number two relates to the "fix" Microsoft have made to avoid infringement of Eolas Technologies' patent "distributed hypermedia method for automatically invoking external application providing interaction and display of embedded objects within a hypermedia document." This means that Internet Explorer displays a message saying "Click to activate and use the control". Surprisingly the music starts playing automatically anyway, but this is still a minor annoyance. The problem was solved by calling this JavaScript function via document.onload():

This complete solution seems to work quite well on all the Windows based browsers I have available now, even Internet Explorer! I have no facility to develop and test this on other platforms, which may not know which plugin to use for the MIME type "application/x-mplayer2". I intend to address this in due course.

In any case the MIME type should strictly speaking be "audio/mpegurl", but this is not supported by Windows. But in case of any difficulty, you can always just click the hypertext link to download the radio.m3u playlist and play it in an external application.

Eolas is a registered trademark of Eolas Technologies Inc. Microsoft, Windows, ActiveX and Internet Explorer are registered trademarks of Microsoft Corporation in the United States and other countries.

I quite like using the 4oD service from Channel 4, I can watch their programs at a time that suits me without the regular ad breaks on live TV. The adverts appear just before the program starts instead, and although you can skip to the right part of a program once it's playing, you can't fast forward the adverts.

The program is delivered to the 4oD player as a play list for Windows Media Player in the ASX file format. The first few items in the play list are the URLs for adverts, and the last one is the actual TV program. To avoid having to sit through these adverts, I saw there was the potential to to intercept this play list and remove the entries pertaining to advertising.

To achieve this, I made use of the redirect_program feature of the Squid web proxy. In this case, it allows me to tell Squid to fetch the play list from my website rather than directly from Channel 4. I wrote the following Perl script based on the example in the Squid documentation:

So when the 4oD client requests a play list from the vodapp.grid.channel4.com server, it instead requests the playlist from my website. My website then downloads the desired play list, and does the required filtering using PHP and XSLT.

This PHP, very simply downloads the play list into a DOM document and applies the the required XSLT to it.

Here's the XSL, that simply copies everything in the document except for any ENTRY that has a TITLE starting with the word advert.

Text size changing underneath mouse pointer

It might seem like a nice idea to have text become bold, or even larger when you position the mouse over it. But this can cause problems when the layout of the page is affected.

Suppose I position my mouse over such text, and it gets bigger. This can cause the text to be moved away from the mouse pointer, which means that the text reverts to its original size. Once the text has shrunk back to its original location under the mouse, it then grows and the cycle repeats until the mouse is repositioned.

My suggestion is to only do this if you can be sure you will not be affecting the layout of the web page. For example make sure the text is bounded within a box that is large enough for the text, regardless of whether it is enlarged by the position of the mouse.

Redirecting to an error page

Occasionally websites have problems, and so it is appropriate to display an error to the user. Rather than having the desired web page display an error, some websites redirect to a dedicated web page that explains an error has occurred.

Errors quite typically are due to a temporary glitch, and so you might expect to be able to use the refresh button in your browser to reload the web page. If you've been redirected to a separate page this isn't possible, even clicking back will redirect you to the error page again.

Instead it seems necessary to click back in rapid succession, before the redirection kicks in. That way you can once again attempt to repeat the steps required to load up the problematic page. This may involve filling in a form once again, but either way this is not user friendly! It would be preferable to display error messages on the web page the error occurred, so that it's possible to simply click refresh.

What country am I in today?

Lots of websites have forms that let select your country. Unfortunately these websites generally have a large number of different countries to choose from. I find this means a regular chore of guessing which country to look for, the right one could be any of the following: Britain, England, Great Britain, United Kingdom or UK. The only option I haven't come across is the actual name of the country where I reside, the United Kingdom of Great Britain and Northern Ireland.

So to anyone writing a web pages with forms allowing a choice of countries. It would be nice to see a default country value determined using a country IP database. That would make things a bit easier.

"Back to" … the future

A lot of sites have links with text such as "Back to homepage", or "Return to to search results". But you may have accessed the page via a link on an external site or even a bookmark. So the wording of such links presumes that a visitor to the site came via the web page that's linked "back" to. My suggestion is to leave out the word "back", as it is fairly redundant and potentially inappropriate.

There are lots of interesting tips and ideas relating to programming techniques and related principals bandied about. The people who advocate them have all sorts of different backgrounds, and undeniably it can be very useful to draw on the experience of others.

Some techniques are devised to solve problems that are common to all programmers alike, but the significance of an underlying problem addressed will vary for each project. While something may be critically important for you, it could well be of insignificant concern to somebody working on a different project.

Each programmer faces difference challenges, and has difference experience - so the solutions they come up with are quite personal to the individual. For example a programmer may advocate a technique that merely addresses a problem specific to certain programming languages, or is specific to the size/complexity of their application or even their development team.

Techniques in computer programming have evolved significantly over time, and opinions are constantly changing. This is no surprise, considering the rapid pace of change in what software is used for and how it is used.

Because of this, I feel it is important to have an awareness of various different techniques. In each new scenario it's then possible to decide what is the most appropriate technique, treating each case on its merits.

Programming is a complex task and difficult to make generalisations about. There are many tips floating around that contradict one another, and have no explanation of what the basis for them is. It is more difficult to evaluate a technique, without some explanation of what its purpose is.

Bad support for font resizing

I use a relatively high resolution 1600*1200, and so most websites look small to me. When I increase the font size in my browser (Firefox) to compensate, the layout of most sites become a complete mess. Typically words are crammed into columns that just look too narrow, or text becomes illegible because it no longer fits inside its containing box. This problem typically occurs because dimensions have been stipulated using pixels in the CSS code, but this will go wrong for other fixed unit of measurement. So when you're coding up your layout using the "box model", try using only em, ex or percentages instead of pixels and the layout should resize in proportion with the text.

Login forms setting focus away from password

Many websites have login pages, which set the focus to the user name field when the page finishes loading. This might sound like a great idea, but not while I'm in the middle of typing in my password! I don't really want half my password in the password box, and the other half plainly visible in the user name field. This is another quite common problem. A simple fix might be to only set focus on the user name field, when it is empty.

Click to enter site

There still seems to be no shortage of sites, with a main page that just says "click to enter site." The fact that such a page actually is the website, seems to be lost on the web designer. Unless there's a good reason for having such an entrance page (like a disclaimer) just let people surf straight in to meaningful content without any such inconvenience.

My homepage has a link to tune into my internet radio presets. This play list file is generated automatically by the web server, which checks whether each of my favourite stations is currently available.

I've updated some of the dead links for some Drum & Bass stations that have moved, and included some new Dubstep stations also. There's now also only one entry in the play list per station, to make channel hopping easier. This also makes the list load up quicker as it does fewer checks.

Give it a try, just click on the link to the file radio.m3u, it should open up in Media Player or iTunes etc. let me know what you think!

The Win32 API methods for the Service Control Manager are powerful but rather unwieldy. Download the Delphi unit service.pas which contains wrapper functions for QueryServiceStatus(), StartService() and ControlService().

• service.pas (4KB)

See the interface section for the unit below, hopefully it's all self explanatory.

The Sudoku page has been improved today. When entering a number, some simple checks are made for an error. If the number entered is OK, it automatically moves to the next cell ready for another number to be typed. I updated the server application also, and the solution can be interactively solved. The server now solves easy parts of the puzzle as you key it in. This is implemented using AJAX programming techniques.

I would also like to set up a daily puzzle. The Pascal source code for the server side solution engine might get posted, if I find time to tidy it up. Please leave comments if you have any ideas!

I thought it'd be a nice idea to include my del.icio.us tagrolls on my homepage. This is made possible by some JavaScript served up by their website. Unfortunately their implementation relies on document.writeln() which is not compatible with XHTML.

Undeterred, I created imposter versions of document.write and document.writeln. My implementations progressively concatenate the parameters into a buffer. When the web page loads, the buffered XHTML is positioned appropriately by using an empty placeholder tag.

I originally tried parsing the buffer as an XML document, with the intention of merging the XML nodes into the XHTML document. I tried using xmlDoc.documentElement.cloneNode(true) as a parameter for appendChild() on an XHTML node.

While this worked like a charm in both Firefox and Opera, Internet Explorer complains of "Interface not supported" and Safari has a fatal error! The failing browsers have an incompatibility of XML and XHTML DOM nodes, even though they both support a common DOM interface.

In the end I had to write my own version of cloneNode() to copy elements between the incompatible schemas and ensure valid nesting of elements. There's an exception handler that fails over to using innerHTML complete with IE kludges just in case.

If you're interested to see how this was achieved, please feel free to view the JavaScript source code.