Bloggy

… that the ~QX11PixmapData(): QPixmap objects must be destroyed before the QApplication object, otherwise the native pixmap object will be leaked. warning most KDE applications display when exiting is actually false.

The X server will cleanup any opened resources, including pixmaps, automatically when the client exits. This is much like how the kernel automatically closes files and memory allocations when an application exits.

(The QPixmaps in question are the ones cached by KIconLoader as best as I can tell. They are cleaned up, just not before QApplication’s destructor runs.)

Posted by mpyne under Computing Troubles & KDE | 5 Comments »

So. Those of you who notice the things I post will presumably notice I haven’t blogged as much recently. Essentially it’s because I have less time for development between school and work (I’m on shore duty so I don’t deploy, but I’m in charge of a division so my hours are still substantial).

The spare time I have had over the past month I’ve mostly spent trying to (re) fix my new favorite bug ever, bug 182026 which is a crash bug in KPixmapCache::discard().

You should hopefully have no clue about what KPixmapCache is other than that it’s what’s crashing most of your favorite apps, so I’ll give you the rundown. Back when KDE 4 was being developed and we were shifting to use the shiny SVG icons and graphics everywhere it was noticed that rendering SVG graphics took rather more time than simply loading a PNG from disk. So, instead of loading a SVG file from disk every single time it was needed, the end result (a PNG) was cached to disk instead, and a quick check is done to ensure that the underlying SVG file didn’t change. I didn’t write the code but thought it was interesting and committed a couple of bug fixes and now I apparently maintain it. =D

For efficiency reasons that on-disk cache is accessed by each KDE process using a shared memory segment (although it then gets inefficient again by using I/O operations anyways, but that’s for later). Anytime you have multiple processes or threads accessing a shared resource you need a way to handle that contention to make sure that the sensitive parts of the code are only being run by one thread at a time.

The second problem that was reported as 182026 was that the “discard” feature of the cache seemed to cause crashes for a lot of people. (Although oddly enough, not for any of the 3 test systems I have. Call me lucky…). The idea of discarding the cache contents is that the application code knows that something changed which means that essentially all of the stored graphics will not be needed. The major way most people see this is by changing Plasma themes (where the old theme’s graphics are now useless) or by changing the icon theme.

The underlying issue is that ::discard() uses essentially no locking. Instead it tells the shared caches to reload themselves on their next operation (a process referred to as “invalidation”), disconnects from shared memory, and deletes the cache file on disk. Or at least, it used to. Merely adding locking helps, but is solving the wrong problem. Any time the cache is disconnected from shared memory, it will try to reconnect later, re-creating the file on disk if necessary (and it is necessary in this case). But this is all just to say that the cache is now empty, there’s no underlying reason that we have to deallocate everything just to turn around and reallocate it.

So when I committed my fix for 182026, I did the following:

• I added a flag to the cache to mark if it was simply empty or not. This allows me to discard the cache without having to disconnect it from shared memory or invalidate it.
• Made KPixmapCache::discard() take the cache lock first before it did any of that.
• When copying the cache for resizing, KSaveFile is used instead of truncating the cache (especially in case any cache locks have to be broken due to timeouts)

Unfortunately my initial fix had the side effect of breaking the “cache” part of KPixmapCache as it would never find cached items again. It was noticed by a KDE Games developer though so 4.4.2 should still be good to go now that I’ve fixed that.

A final side note is that I think I need to change the code that deletes an entire named cache to discard the cache as well for those processes that already have it attached to shared memory.

I have an alternative shared-memory cache implementation which I hope to integrate for KDE Platform 4.5 or 4.6 (although it may simply end up as KSharedCache or similar instead of merely replacing KPixmapCache due to API additions in KPixmapCache I don’t feel like supporting). At some point in between I may write a post about the underlying cache architecture and some “Do”s and “Don’t”s but I think I’ll just stop here for now. :P

Posted by mpyne under C++ & Computing Troubles & KDE & Programming | 3 Comments »

The “fun” in the title should be read in your most sarcastic tone of voice… Anyways, one of my professors mailed us a PDF of a scanned document to read (and print out) for the next class. Being that is was scanned in (by what appeared to be the professor literally holding it above a scanner) there was a lot of excess black in the picture.

I don’t know about you, but printing 2 large blocks of solid black, for 22 pages, doesn’t sound like a wise investment of toner. But ah! Why don’t I just crop off the excess part of each page so that just the scanned-in text is visible, and print that out? This has to be easy, right?

Unfortunately it wasn’t as easy as I’d hoped (most of the picture editors that can even handle PDFs can’t print out each layer as a separate page, and there’s no way I’m doing the exact same operation 22 times). ImageMagick looked like the thing I needed, even if it would take some trial-and-error to figure out exactly how much to crop off.

Turned out it only took a couple of runs to figure out exactly how much I could get away with cropping. But I had a worse problem than having to do trial runs: The output looked horrible.

I tried reading the man page, going to the website, and the rest, and couldn’t figure out what to do. Using the -density option seemed to be the right idea, but alas I couldn’t get it to work.

I troubleshot further, even getting to the point of running gs manually to see if Ghostview or ImageMagick was the problem (turned out it was myself, I guess). Eventually I realized that Ghostview was rendering the initial image to ImageMagick at a low resolution (72 DPI) but viewing the source in Okular, it was obvious that much better was possible (I’d estimate 200 DPI although I ended up using 300). So if I could figure out how to get ImageMagick to pass the right DPI to Ghostview I should have the problem fixed.

More directed Google searching revealed I’d had the right flag the whole time, -density. I just had it in the wrong spot. Something like this is right: convert -density 300x300 input.pdf -crop ... output.pdf. Instead I’d been using convert input.pdf -density 300x300 -crop ... output.pdf.

I figured I’d put my experience out there in the great Internet Memory Machine in case others have similar troubles.

Posted by mpyne under Computing Troubles & Useful Tricks | 10 Comments »

So, in The Beginning, when I was just a young padawan on the Internet, I had been let into a glorious secret: Emulation (not of IBM System/360 machines, but of more important things like the Super NES). Some branching from there led me to zophar.net, a popular emulation site, and their message boards, and also left me with a fascination with emulation.

The attributes of some of the older systems like the NES and Super NES made it fairly easy to capture their music-producing software, since those systems used separate co-processors to handle music effects. NES music would be stored in the NSF format, and SNES music was handled with the SPC format (named after the audio chip used, the Sony SPC700). There were (and still are) specialized plugins on many systems to play these formats (they emulated only the music chip, not the rest of the system).

I’ve been involved on the periphery of some of these things for the past couple of years. (For instance I had written a KFileMetaInfo plugin for KDE 3, and had helped Chris Lee with adding playback support to GStreamer.

One problem with the previous GStreamer solution (which I’ll call gst-spc) is that the underlying playback library, libopenspc, is written in x86 assembly, and has some crash bugs associated with it as well. As well the code has long been orphaned. I’m not really any good at writing emulation code and although I could learn, it would take far too much time for me to do anything useful.

Luckily for me the state of the art has advanced and last year I was pointed to a library called game-music-emu. This library included a very good SPC emulator written in C++, which had been merged into some popular SNES emulators already. Unfortunately it didn’t really have a great build system (using it involved simply copying it into your existing program) so my initial proposal to port GStreamer to use game-music-emu by simply including the source files with GStreamer was rejected. The GStreamer devs preferred to have an external library which could be used (or not) and I couldn’t blame them since in general good OSS projects avoid copying or forking external code.

So I contacted the game-music-emu author (Blargg) asking about the possibility of adding support for building a library, and ended up with commit access and an invitation to do it myself. Hmm.

So I did, and awhile ago I had made a release of “libgme” 0.5.5, working with Blargg has he got free time. My subsequent patch to GStreamer was accepted and since gst-plugins-bad-0.10.14 it has been possible to use libgme to playback many emulated music file types (not just SNES, but others as well).

With that solved I left the issue, but I recently came back to it since I figured out that even after upgrading to gst-plugins-bad-0.10.17 the other day, that gstreamer playback was not using libgme, but the older libopenspc.

At first I thought it was simply my fault, as I’d still had gst-spc installed from years and years ago. Removing gst-spc and libopenspc (just to be double-sure) left me with no SPC playback features. Running gst-inspect confirmed I did not have any gme decoder. WTF.

I then again thought it was my fault because I had installed libgme to /usr/local instead of /usr. So I dutifully wrapped up libgme in an ebuild and installed it. And still nothing. WTF.

I dug into the Gentoo ebuild for gst-plugins-bad and it seems that for whatever reason not all possible plugins are installed. It seems the new installation method is supposed to be that each individual plugin is supposed to have its own ebuild (i.e. gst-plugins-gme), like how Gentoo has split out other packages like KDE into individual ebuilds. Fair enough.

I write another ebuild, and finally hit paydirt:

The Qt example music player playing SPC files

Obviously this does require that you are using the GStreamer backend for Phonon to have this work, otherwise you can just try it in some other GStreamer-using application. (I’d show it in JuK but I’d have to add SPC support to Taglib first)

If you’re interested in the ebuilds I used you can use this Portage overlay, (SHA-512 sum c0ff9aa5413b0c0b14f7c52d5b3ee887edc4e7bf47182e58c21e9c340d8ff7e9). The overlay may or may not work for you, and I don’t even know if overlays are still the “hip” way to do things in Gentoo, but It Works For Me. ;)

Posted by mpyne under Computing Troubles & KDE & Personal & Programming | 3 Comments »

So if you’ve used kdesvn-build to build some of the modules that are hosted on Gitorious then you are probably familiar with an error that always comes up when doing the initial checkout. This error is so famous that every “how to build using kdesvn-build” guide I’ve seen over the past couple of months have mentioned that the clone step for qt-copy would need to be done manually.

A Konversation developer, argonel, noticed the issue the other day and got in touch with me, so I had him strace the output of the (successful) manual run and the (unsuccessful) kdesvn-build run. It wasn’t initially super helpful although it clarified what was going on (the gitorious.org end of the connect was closed on their end for some reason).

That was the conclusion of that, but then I get an email the next day from argonel saying that he’d done more digging, and that it was a known issue which could be worked around by adding the -v flag to git, which forces progress output to be displayed even if the output is redirected. (The issue has something to do with kdesvn-build redirecting the git output, if you run the git command manually but redirect its stdout to a file then you’ll see the clone fail after about 30 seconds as well).

This progress output makes the logged output look really bad, however, so the workaround I ended up implementing in kdesvn-build is to show the progress output on the terminal and redirect the rest (you may actually prefer this as it’s possible to see the progress of the checkout now).

In short, the Great kdesvn-build Git Clone Bug should be fixed. Please test the trunk version of kdesvn-build for me to make sure I got it though!

Posted by mpyne under Computing Troubles & KDE & kdesvn-build | 5 Comments »

So I woke up this morning to notice that my kdesvn-build --refresh-build run from last night had a hiccup somewhere:

The problem was kdelibs failed to install, which meant every subsequent module error-ed out during the CMake process. The problem with kdelibs was related to me having an old library installed in my Qt directory since I hadn’t cleared out my Qt/KDE installation directories first (whoops).

It would be really annoying to have to type out each module to build just to skip the first four that had built right. Luckily I don’t have to do that, instead this sufficed:

$ kdesvn-build --no-src --resume-from kdepimlibs
Script started processing at Sat Dec 12 19:04:24 2009
< <<  Build Process  >>>
Building kdepimlibs (1/21)
        Preparing build system for kdepimlibs.
        Removing files in build directory for kdepimlibs
        ...

The magic here is the --resume-from command line option which skips until the given module in the build process. --no-src is only available since kdesvn-build-1.10, use --no-svn if you haven’t upgraded yet.

Posted by mpyne under Computing Troubles & KDE & Useful Tricks & kdesvn-build | 1 Comment »

If you’re a KDE developer using a recent version of glibc (since 2.10), you may have come across strange crashes complaining about memory corruption when running development versions of KDE (especially Okular, KTorrent, KNotify, and other KDE applications using threading).

This had been noticed by quite a few people (and was being tracked by KDE under bug 196207). Apparently OpenSUSE had noticed this as well and had submitted a patch to the glibc maintainers this past June, in sourceware bug 10282. I am happy to be able to say that the patch works for me™. So if you’re been having this issue, see about getting your distribution to update their glibc with this patch (or a better one, I don’t care ;)

Posted by mpyne under Computing Troubles & KDE & Useful Tricks | 1 Comment »

As usual, no (coding) progress happens for me unless something else is causing me problems on my system. In this case, I had the opportunity to need to use my library dependency viewer.

Qt is nice enough to link to KIO when run under a KDE desktop, so even though ELF Library Viewer only links to Qt right now, I still get the nice KDE file open dialog. This is 99% of the time a positive feature. But for some reason the dialog was crashing and just having a horrible time handling large directories (like, say, /usr/bin). Worse yet, I couldn’t simply type /usr/bin/foo and click OK, since that isn’t working either. Hmm.

I decided the easiest thing to do would be to add support for opening files passed to the command line of my application (and I was right). Since I was working on it anyways I also ported the build system to CMake (from QMake), and so now it’s possible to install it/make packages/etc.

I’ve placed the updated version on my software page in case anyone’s interested. Obligatory screenshot is the same as two years ago:

Now it’s back to reading research papers all day. :-/

Posted by mpyne under Computing Troubles & KDE & Programming | 2 Comments »

From IRC, a quote:

Son, we live in a world that has bugs, and those bugs have to be found by men with debuggers. Who’s gonna do it? You? You, OSS fan boy? I have a greater responsibility than you could possibly fathom…. You have the luxury of not knowing what I know. That the KDE 4.0 release, while tragic, probably saved developer resources. And that my existence, while grotesque and incomprehensible to you, saves developer resources. You don’t want the truth because deep down in places you don’t talk about at parties, you want me on that bug, you need me on that bug.

And in other news, I’m officially on shore duty, at my next duty station. I got to “help” the nice Comcast guy with hooking up cable Internet with linux, but other than that things went smoothly. (Protip:Have a DNS nameserver not part of Comcast memorized before hooking up the Internet. If you’re unlucky you’ll get to a point where your Internet works but DNS does not.)

Posted by mpyne under Computing Troubles & Funny & KDE & Useful Tricks | 2 Comments »

So after my latest X.org upgrade I forgot to recompile the xf86-input-evdev module as well. So of course when I ran startx again, my shiny KDE desktop came straight up… and I couldn’t move the mouse or use the keyboard. Drat.

Now I’ve been much more careful to cleanly shutdown KDE when possible ever since I started losing rc files with my previous filesystem. And besides which, I couldn’t use Ctrl-Alt-Backspace even if I wanted. So what I did instead was login with my laptop to try and shutdown KDE.

Of course, there’s still the tricky issue of how to cleanly shutdown your running session from a completely different shell. Luckily ksmserver (the process responsible for performing the shutdown) has a DBus interface:

kde-svn@midna ~ $ qdbus org.kde.ksmserver /KSMServer
method bool org.kde.KSMServerInterface.canShutdown()
method QString org.kde.KSMServerInterface.currentSession()
method void org.kde.KSMServerInterface.logout(int, int, int)
method void org.kde.KSMServerInterface.resumeStartup(QString)
snip

Of course, that’s what I get when I run if from my current KDE session. I was worried that in my ssh shell there would be problems with accessing the D-Bus session. So imagine my surprise when I was wrong, the command worked just fine even with no environment variable set. Huh.

Although it saved me from what I was going to do (look through /proc/pid/environ for a process that I knew was in the DBus session), it was still unexpected to me. Turns out that DBus will, if the environment variable is unset, try to “autolaunch” the session bus. This entails looking in the X session and in ~/.dbus/session-bus/ to look for the information on the DBus session, and if not able to locate the session, to simply launch another DBus session. In my case the information was available in ~/.dbus/session-bus/.

So I’m impressed, I was expecting it to be a giant pain in the ass and it was actually ridiculously simple.

Btw, the conclusion to the tale was: qdbus org.kde.ksmserver /KSMServer org.kde.KSMServerInterface.logout 0 0 0. The three parameters were annoying for me to lookup so here it is:

The real signature: KSMServerInterface::logout( KWorkSpace::ShutdownConfirm, KWorkSpace::ShutdownType, KWorkSpace::ShutdownMode ). The parameters are essentially the same as in KWorkSpace::canShutdown(), and the values are defined on that API page as well. One thing to note is that the final 0 (shutdown mode) is actually useless since we’re not requesting that a shutdown happen, only a logout. If we were requesting a shutdown, the shutdown mode is what selects whether to force a shutdown/reboot even if other people were also logged in, for instance.

Kind of as an aside, I’ve taken the liberty of updating my GPG keys due to the relative insecurity and low strength of the 1024-bit DSA key I had before. It should have a fingerprint of 5406 ECE8 3665 DA9D 201D 3572 0BAF 0C9C 7B6A E9F2, and is available here. Maybe someday I’ll even be able to attend a keysigning event to get it signed by someone else. :-/

Posted by mpyne under Computing Troubles & KDE | 4 Comments »