Bloggy

The “fun” in the title should be read in your most sarcastic tone of voice… Anyways, one of my professors mailed us a PDF of a scanned document to read (and print out) for the next class. Being that is was scanned in (by what appeared to be the professor literally holding it above a scanner) there was a lot of excess black in the picture.

I don’t know about you, but printing 2 large blocks of solid black, for 22 pages, doesn’t sound like a wise investment of toner. But ah! Why don’t I just crop off the excess part of each page so that just the scanned-in text is visible, and print that out? This has to be easy, right?

Unfortunately it wasn’t as easy as I’d hoped (most of the picture editors that can even handle PDFs can’t print out each layer as a separate page, and there’s no way I’m doing the exact same operation 22 times). ImageMagick looked like the thing I needed, even if it would take some trial-and-error to figure out exactly how much to crop off.

Turned out it only took a couple of runs to figure out exactly how much I could get away with cropping. But I had a worse problem than having to do trial runs: The output looked horrible.

I tried reading the man page, going to the website, and the rest, and couldn’t figure out what to do. Using the -density option seemed to be the right idea, but alas I couldn’t get it to work.

I troubleshot further, even getting to the point of running gs manually to see if Ghostview or ImageMagick was the problem (turned out it was myself, I guess). Eventually I realized that Ghostview was rendering the initial image to ImageMagick at a low resolution (72 DPI) but viewing the source in Okular, it was obvious that much better was possible (I’d estimate 200 DPI although I ended up using 300). So if I could figure out how to get ImageMagick to pass the right DPI to Ghostview I should have the problem fixed.

More directed Google searching revealed I’d had the right flag the whole time, -density. I just had it in the wrong spot. Something like this is right: convert -density 300x300 input.pdf -crop ... output.pdf. Instead I’d been using convert input.pdf -density 300x300 -crop ... output.pdf.

I figured I’d put my experience out there in the great Internet Memory Machine in case others have similar troubles.

Posted by mpyne under Computing Troubles & Useful Tricks | 10 Comments »

So, in The Beginning, when I was just a young padawan on the Internet, I had been let into a glorious secret: Emulation (not of IBM System/360 machines, but of more important things like the Super NES). Some branching from there led me to zophar.net, a popular emulation site, and their message boards, and also left me with a fascination with emulation.

The attributes of some of the older systems like the NES and Super NES made it fairly easy to capture their music-producing software, since those systems used separate co-processors to handle music effects. NES music would be stored in the NSF format, and SNES music was handled with the SPC format (named after the audio chip used, the Sony SPC700). There were (and still are) specialized plugins on many systems to play these formats (they emulated only the music chip, not the rest of the system).

I’ve been involved on the periphery of some of these things for the past couple of years. (For instance I had written a KFileMetaInfo plugin for KDE 3, and had helped Chris Lee with adding playback support to GStreamer.

One problem with the previous GStreamer solution (which I’ll call gst-spc) is that the underlying playback library, libopenspc, is written in x86 assembly, and has some crash bugs associated with it as well. As well the code has long been orphaned. I’m not really any good at writing emulation code and although I could learn, it would take far too much time for me to do anything useful.

Luckily for me the state of the art has advanced and last year I was pointed to a library called game-music-emu. This library included a very good SPC emulator written in C++, which had been merged into some popular SNES emulators already. Unfortunately it didn’t really have a great build system (using it involved simply copying it into your existing program) so my initial proposal to port GStreamer to use game-music-emu by simply including the source files with GStreamer was rejected. The GStreamer devs preferred to have an external library which could be used (or not) and I couldn’t blame them since in general good OSS projects avoid copying or forking external code.

So I contacted the game-music-emu author (Blargg) asking about the possibility of adding support for building a library, and ended up with commit access and an invitation to do it myself. Hmm.

So I did, and awhile ago I had made a release of “libgme” 0.5.5, working with Blargg has he got free time. My subsequent patch to GStreamer was accepted and since gst-plugins-bad-0.10.14 it has been possible to use libgme to playback many emulated music file types (not just SNES, but others as well).

With that solved I left the issue, but I recently came back to it since I figured out that even after upgrading to gst-plugins-bad-0.10.17 the other day, that gstreamer playback was not using libgme, but the older libopenspc.

At first I thought it was simply my fault, as I’d still had gst-spc installed from years and years ago. Removing gst-spc and libopenspc (just to be double-sure) left me with no SPC playback features. Running gst-inspect confirmed I did not have any gme decoder. WTF.

I then again thought it was my fault because I had installed libgme to /usr/local instead of /usr. So I dutifully wrapped up libgme in an ebuild and installed it. And still nothing. WTF.

I dug into the Gentoo ebuild for gst-plugins-bad and it seems that for whatever reason not all possible plugins are installed. It seems the new installation method is supposed to be that each individual plugin is supposed to have its own ebuild (i.e. gst-plugins-gme), like how Gentoo has split out other packages like KDE into individual ebuilds. Fair enough.

I write another ebuild, and finally hit paydirt:

The Qt example music player playing SPC files

Obviously this does require that you are using the GStreamer backend for Phonon to have this work, otherwise you can just try it in some other GStreamer-using application. (I’d show it in JuK but I’d have to add SPC support to Taglib first)

If you’re interested in the ebuilds I used you can use this Portage overlay, (SHA-512 sum c0ff9aa5413b0c0b14f7c52d5b3ee887edc4e7bf47182e58c21e9c340d8ff7e9). The overlay may or may not work for you, and I don’t even know if overlays are still the “hip” way to do things in Gentoo, but It Works For Me. ;)

Posted by mpyne under Computing Troubles & KDE & Personal & Programming | 3 Comments »

So if you’ve used kdesvn-build to build some of the modules that are hosted on Gitorious then you are probably familiar with an error that always comes up when doing the initial checkout. This error is so famous that every “how to build using kdesvn-build” guide I’ve seen over the past couple of months have mentioned that the clone step for qt-copy would need to be done manually.

A Konversation developer, argonel, noticed the issue the other day and got in touch with me, so I had him strace the output of the (successful) manual run and the (unsuccessful) kdesvn-build run. It wasn’t initially super helpful although it clarified what was going on (the gitorious.org end of the connect was closed on their end for some reason).

That was the conclusion of that, but then I get an email the next day from argonel saying that he’d done more digging, and that it was a known issue which could be worked around by adding the -v flag to git, which forces progress output to be displayed even if the output is redirected. (The issue has something to do with kdesvn-build redirecting the git output, if you run the git command manually but redirect its stdout to a file then you’ll see the clone fail after about 30 seconds as well).

This progress output makes the logged output look really bad, however, so the workaround I ended up implementing in kdesvn-build is to show the progress output on the terminal and redirect the rest (you may actually prefer this as it’s possible to see the progress of the checkout now).

In short, the Great kdesvn-build Git Clone Bug should be fixed. Please test the trunk version of kdesvn-build for me to make sure I got it though!

Posted by mpyne under Computing Troubles & KDE & kdesvn-build | 5 Comments »

So I woke up this morning to notice that my kdesvn-build --refresh-build run from last night had a hiccup somewhere:

The problem was kdelibs failed to install, which meant every subsequent module error-ed out during the CMake process. The problem with kdelibs was related to me having an old library installed in my Qt directory since I hadn’t cleared out my Qt/KDE installation directories first (whoops).

It would be really annoying to have to type out each module to build just to skip the first four that had built right. Luckily I don’t have to do that, instead this sufficed:

$ kdesvn-build --no-src --resume-from kdepimlibs
Script started processing at Sat Dec 12 19:04:24 2009
< <<  Build Process  >>>
Building kdepimlibs (1/21)
        Preparing build system for kdepimlibs.
        Removing files in build directory for kdepimlibs
        ...

The magic here is the --resume-from command line option which skips until the given module in the build process. --no-src is only available since kdesvn-build-1.10, use --no-svn if you haven’t upgraded yet.

Posted by mpyne under Computing Troubles & KDE & Useful Tricks & kdesvn-build | 1 Comment »

If you’re a KDE developer using a recent version of glibc (since 2.10), you may have come across strange crashes complaining about memory corruption when running development versions of KDE (especially Okular, KTorrent, KNotify, and other KDE applications using threading).

This had been noticed by quite a few people (and was being tracked by KDE under bug 196207). Apparently OpenSUSE had noticed this as well and had submitted a patch to the glibc maintainers this past June, in sourceware bug 10282. I am happy to be able to say that the patch works for me™. So if you’re been having this issue, see about getting your distribution to update their glibc with this patch (or a better one, I don’t care ;)

Posted by mpyne under Computing Troubles & KDE & Useful Tricks | 1 Comment »

As usual, no (coding) progress happens for me unless something else is causing me problems on my system. In this case, I had the opportunity to need to use my library dependency viewer.

Qt is nice enough to link to KIO when run under a KDE desktop, so even though ELF Library Viewer only links to Qt right now, I still get the nice KDE file open dialog. This is 99% of the time a positive feature. But for some reason the dialog was crashing and just having a horrible time handling large directories (like, say, /usr/bin). Worse yet, I couldn’t simply type /usr/bin/foo and click OK, since that isn’t working either. Hmm.

I decided the easiest thing to do would be to add support for opening files passed to the command line of my application (and I was right). Since I was working on it anyways I also ported the build system to CMake (from QMake), and so now it’s possible to install it/make packages/etc.

I’ve placed the updated version on my software page in case anyone’s interested. Obligatory screenshot is the same as two years ago:

Now it’s back to reading research papers all day. :-/

Posted by mpyne under Computing Troubles & KDE & Programming | 2 Comments »

From IRC, a quote:

Son, we live in a world that has bugs, and those bugs have to be found by men with debuggers. Who’s gonna do it? You? You, OSS fan boy? I have a greater responsibility than you could possibly fathom…. You have the luxury of not knowing what I know. That the KDE 4.0 release, while tragic, probably saved developer resources. And that my existence, while grotesque and incomprehensible to you, saves developer resources. You don’t want the truth because deep down in places you don’t talk about at parties, you want me on that bug, you need me on that bug.

And in other news, I’m officially on shore duty, at my next duty station. I got to “help” the nice Comcast guy with hooking up cable Internet with linux, but other than that things went smoothly. (Protip:Have a DNS nameserver not part of Comcast memorized before hooking up the Internet. If you’re unlucky you’ll get to a point where your Internet works but DNS does not.)

Posted by mpyne under Computing Troubles & Funny & KDE & Useful Tricks | 2 Comments »

So after my latest X.org upgrade I forgot to recompile the xf86-input-evdev module as well. So of course when I ran startx again, my shiny KDE desktop came straight up… and I couldn’t move the mouse or use the keyboard. Drat.

Now I’ve been much more careful to cleanly shutdown KDE when possible ever since I started losing rc files with my previous filesystem. And besides which, I couldn’t use Ctrl-Alt-Backspace even if I wanted. So what I did instead was login with my laptop to try and shutdown KDE.

Of course, there’s still the tricky issue of how to cleanly shutdown your running session from a completely different shell. Luckily ksmserver (the process responsible for performing the shutdown) has a DBus interface:

kde-svn@midna ~ $ qdbus org.kde.ksmserver /KSMServer
method bool org.kde.KSMServerInterface.canShutdown()
method QString org.kde.KSMServerInterface.currentSession()
method void org.kde.KSMServerInterface.logout(int, int, int)
method void org.kde.KSMServerInterface.resumeStartup(QString)
snip

Of course, that’s what I get when I run if from my current KDE session. I was worried that in my ssh shell there would be problems with accessing the D-Bus session. So imagine my surprise when I was wrong, the command worked just fine even with no environment variable set. Huh.

Although it saved me from what I was going to do (look through /proc/pid/environ for a process that I knew was in the DBus session), it was still unexpected to me. Turns out that DBus will, if the environment variable is unset, try to “autolaunch” the session bus. This entails looking in the X session and in ~/.dbus/session-bus/ to look for the information on the DBus session, and if not able to locate the session, to simply launch another DBus session. In my case the information was available in ~/.dbus/session-bus/.

So I’m impressed, I was expecting it to be a giant pain in the ass and it was actually ridiculously simple.

Btw, the conclusion to the tale was: qdbus org.kde.ksmserver /KSMServer org.kde.KSMServerInterface.logout 0 0 0. The three parameters were annoying for me to lookup so here it is:

The real signature: KSMServerInterface::logout( KWorkSpace::ShutdownConfirm, KWorkSpace::ShutdownType, KWorkSpace::ShutdownMode ). The parameters are essentially the same as in KWorkSpace::canShutdown(), and the values are defined on that API page as well. One thing to note is that the final 0 (shutdown mode) is actually useless since we’re not requesting that a shutdown happen, only a logout. If we were requesting a shutdown, the shutdown mode is what selects whether to force a shutdown/reboot even if other people were also logged in, for instance.

Kind of as an aside, I’ve taken the liberty of updating my GPG keys due to the relative insecurity and low strength of the 1024-bit DSA key I had before. It should have a fingerprint of 5406 ECE8 3665 DA9D 201D 3572 0BAF 0C9C 7B6A E9F2, and is available here. Maybe someday I’ll even be able to attend a keysigning event to get it signed by someone else. :-/

Posted by mpyne under Computing Troubles & KDE | 4 Comments »

If you’ve been using kdesvn-build to build qt-copy, I recommend updating to the Subversion /trunk version, or using the 1.9.1 release. This is to adapt to a recent change to the qt-copy configure script, which causes an infinite loop (with subsequent disk fillup thanks to output logging). 1.9.1 will not work with older versions of qt-copy however.

For those who are interested, the change is to add a method for accepting a license without user prompting, which means that the previous method kdesvn-build employed (essentially running sed on the configure script) is now unnecessary (and harmful :-/).

In other KDE news, I ran across an issue with KWin closing on me after I closed out of all applications, starting from the other day. I’ve committed a fix to trunk, but I’m still not sure why it’s started exactly. But if you’re running into issues with KWin closing on you when you close all windows make sure to update kdebase.

Posted by mpyne under Computing Troubles & KDE & kdesvn-build | No Comments »

So the other day I had suffered probably the third dataloss in a year due to the XFS filesystem in use on my desktop + power loss.

Update: I just realized that I failed to mention that the dataloss is my fault, not XFS’s. Had I done more diligence in my research when I set it up in the first place I would have been aware of that particular shortcoming (shared with ext4 in its 2.6.29 and earlier configuration) where metadata updating before data, merged with powerloss could result in zero-length files. The files I lost were always ones being written to at the time, never just “random” files. Anyways…

I had already setup fairly well I thought, given that I was using RAID 1 (i.e. two redundant hard drives to hold one hard drive of data) and had UPS protection on the computer anyways.

The failures in my setup were notable though:

1. My backup strategy depended on me having enough time to actually hook up the external drive, run the backup, unhook it, etc. Scripting it so that it would take less time still required too much time to initially setup. Therefore my backups were fairly out of date. Compounding this, when I finally did setup the external and try to perform a backup, I found to my displeasure that it was ntfs formatted and therefore I couldn’t write to it. (I had the read/write ntfs-3g driver available but had forgotten about it by then, and didn’t have time to troubleshoot further).
2. UPS only works when your 2 year old son doesn’t go hard resetting the power on you.
3. UPS + disabling the front power switch only saves you until your next kernel panic induced by bleeding-edge video drivers (the fact that they were open source didn’t save me here ;)
4. The RAID worked — both hard drives were in the same inconsistent state, with the same lost files after the fsck… :(

The corrective action for all of this is complete however. Last night I used ntfsresize on the external drive (since it had data I wanted to retain) and used the free space to make a new ext3 partition for backups. From there I booted up a Gentoo LiveCD and copied all of the hard drive’s data to the external drive. I then reformatted the hard drives with ext4 (with appropriate mount options to avoid having it cause the same issues I have now with xfs) and copied all the data back.

There were some hiccups relating to me using manual mdadm commands to bring the hard drives online from the LiveCD. I apparently changed some of the md device names on the hard drives in the process which required some more manual mdadm –assemble action to fix. But everything is working so far, although I’m sure there’s a few packages that need to be reinstalled to account for the dataloss over time.

Luckily the major reason for my lack of free time looks like it will be rectified within the next week, more on that later.

Posted by mpyne under Computing Troubles & Personal | 14 Comments »