Bloggy

I finally fixed a bug that has been pestering me for literally months. It is bug 117541. The story of how I finally tracked down the cause may be interesting, and should certainly be a cautionary tale for C++ programmers. Allow me to relate the story. (Less technically inclined readers: I will try to make it understandable, but this is a bit dry)

I first noticed the bug when I upgraded some packages on my system around May 2005. The upgrades included an upgrade to glibc (the Gnu C library, one of the core pieces the system is built around). The upgrades included a handy feature to detect programs which were using memory incorrectly, and to abort them pre-emptively (since it could be an attack that causes the memory corruption). The problem was that when I ran JuK from Konsole (where you can see the debugging output), it would always display a glibc crash message when I shutdown JuK. Actually, JuK would always crash, but normally you wouldn’t see the message, and the nice little “Your program crashed” window would never show up for some reason, so I didn’t notice before.

I poked around for a bit, but I couldn’t discover the cause. This was when I was working on the improved cover art support for JuK, and one thing that was necessary was to save covers on shutdown. This crash was preventing that from happening, so I forced the covers to save before the shutdown crash seemed to happen since I couldn’t seem to find the bug.

Since the bug wasn’t exactly serious, and since it seemed to be only affecting me (I was the only one who seemed to get the crash), I just let it go. I didn’t even report it in the KDE Bug database.

A few days ago however, I got a report from a couple of different sources that they were noticing crashes on shutdown. And with the same error message I got! Now that I knew I wasn’t the only one getting the bug, I decided to try again.

First, I realized I needed to bring out the big guns for debugging. I installed the latest valgrind (and disabled the exec-stack protection which somehow found its way on my system :). Now, valgrind is nice because it is a tool that can track each and every single bit of memory your application uses, and tell you exactly how it is incorrectly used. Better yet, it will tell you from what part of the code the error occurred.

I reproduced the bug under valgrind, and received the following output:

JuK valgrind log

So what was happening is that the PlaylistItem::Pointer class, which guards the PlaylistItem, was trying to write to memory it no longer had access to. And then, even worse, JuK was trying to free memory that had already been freed.

But looking at the PlaylistItem::Pointer code, it looked to be perfect. There were certainly no obvious bugs where valgrind was pointing.

So what I did was start to add a debugging harness around PlaylistItem::Pointer. I even went so far as to add a function to JuK’s DCOP interface to allow me to print out the status of the PlaylistItem::Pointer internal data structures at any time.

This helped a little bit. I discovered that there were very many PlaylistItems that were being pointed to, but didn’t know that they were being pointed to.

The “obvious” thing to do from there is to figure out what was causing the PlaylistItem to reset and think it wasn’t being guarded. But there were only a few lines in all of JuK that could cause such a reset, and they all looked very innocuous. There was PlaylistItem::Pointer, which as I mentioned was damn near perfect. The only other option was the PlaylistItem construction code. But that code is only run once, and runs before the PlaylistItem::Pointer starts to point at it! There should have been no way to get the results I was getting.

I was stumped, even though I could have figured it out here. What I ended up doing was adding even more debugging. I would add calls to every function in PlaylistItem::Pointer to make sure the data was still consistent. When the data became inconsistent, it would force a crash. Eventually I was able to get JuK to crash in the very function call that made the data inconsistent.

What I discovered was surprising: The PlaylistItem constructor really was being run after its PlaylistItem::Pointer was made. Huh?

It made sense when I saw the backtrace of calls leading to the failure though. This had occurred while the PlaylistItem was being created. What happens is that when a PlaylistItem is created, it first has to call some code that it inherits from Qt and KDE. This happens before the PlaylistItem initializes its own data.

The Qt code (in QListViewItem) takes care of adding the newly created item into its parent list on-screen, using a function called insertItem() in the parent. JuK had its own implementation of that function, which was used to wrap the new item in a PlaylistItem::Pointer and then add it to a list of items to process when the results were needed. This way items could be added quickly, even if the user deleted the item in the meantime before the result was needed. But, this was happening from the Qt code. The PlaylistItem constructor hadn’t been run yet.

So what happened is that the new PlaylistItem::Pointer correctly set a flag in the new item to indicate that it was being guarded. Then, once insertItem() and the rest of Qt were done, the PlaylistItem constructor finally ran, and accidentally reset that flag to indicate no guards. Eventually the PlaylistItem was deleted, and since it had no guards, it didn’t indicate to the PlaylistItem::Pointer mechanism that it needed to be cleaned up.

On shutdown, JuK’s remaining PlaylistItem::Pointer objects would try to delete the items they were pointing to, but they had already been deleted without being cleaned up. This caused the two error messages detected in valgrind. This same error was detected by glibc, which forced the crash.

The code had to defeat a C++ mechanism to stop this kind of error, by forcing the code to pretend that the QListViewItem was a PlaylistItem using static_cast. If we had used dynamic_cast instead, which checks if the conversion is valid during program execution, we would have caught the error much sooner.

So the moral of the story is, if a QListView method you’re overridding says that it uses a QListViewItem*, that is possibly true even if you never create a QListViewItem* directly! Be careful, and use dynamic_cast when possible.

Posted by mpyne under Uncategorized | No Comments »

I realized that even after all these years of having one web prescence or another, that I don’t have a page that people can look at and say, “Oh, that’s his email address”. I’ve rectified that, which should be especially useful for my family members who read this and share my propensity for not remembering to record important email addresses. ;-)

I think the reason I have left it go for so long is that I’m afraid spammers will find the address and force me to finally setup SpamAssassin or something. That’s right, I still don’t have spam filtering software setup. :-)

To pre-emptively counteract the threat, I have encoded the address using HTML entities instead of just typing it out. I also broke up the mailto: link using JavaScript (the e-mail address will still show if JavaScript is disabled though). So here’s hoping that the spammer’s email address harvesting tools are not standards-compliant. ;-)

Computer-related things have been slower for myself recently, for a few reasons (Coding fatigue, my Nintendo fatigue going away, etc.). I’m kinda depressed about it because one of the bugs I’ve been trying to fix keeps evading me. It’s bug 116181 in case anyone is interested, apparently we don’t know how to use the KHTML library. Odd thing is that this worked in KDE 3.4, and it briefly worked in KDE 3.5, and now I can’t figure out why it crashes for some people and it doesn’t for others.

Anyways, time to spend more time away from the computer. One of the bigger Steelers games in recent memory is happening tonight. Can they defeat the juggernaut, the “unstoppable” Colts?

Posted by mpyne under Uncategorized | No Comments »

Did well on the last test (which was the last non-final exam). I actually scored above my average, which is nice since my study hours have been slowly decreasing while the difficulty has been ramping up, so I haven’t scored above my average for about 5 straight tests. Although I haven’t been doing poorly either.

If you have been listening to CDs from Sony (which owns BMG Records among other labels) on your Windows computer, and the CD is from within about the last year, your computer is probably infected with a rather nasty rootkit.

A rootkit is software that takes control of the very inner portions of your operating system, and is pretty much always an insidious event. In this case, the rootkit is used to prevent Windows from running different applications, which could possibly be used to copy the CD. It is also used to hide the fact that it is present and installed on the system. Attempting to remove the rootkit by yourself is likely to break your system.

I’m not sure how Sony had the audacity to actually do this though. If I were to infect 500,000 computers with this rootkit, my tail would probably be getting hauled off right this instant to the nearest Federal “Don’t drop the soap!” prison.

In case you’re interested in how this was discovered, this is a good source of information: http://www.boingboing.net/2005/11/14/sony_anticustomer_te.html. Also good is this writeup by LWN (subscription required): http://lwn.net/Articles/160023/.

The ironic thing? The software in question is designed to try and prevent copyright infringement, but it contains at least 3 different libraries/programs in violation of their license. Seems like the pot calling the kettle black.

Even better is this quote by a high-ranking officer at Sony: “Most people, I think, don’t even know what a rootkit is, so why should they care about it?” — Thomas Hesse, SonyBMG Manager.

By the same reasoning, it would be OK for me to infect your system using a buffer overflow, but only if you don’t know what a buffer overflow is. I don’t buy that reasoning, and neither do the courts.

In funnier news, this comic, describing the software design process is pretty hilarious.

Posted by mpyne under Uncategorized | No Comments »

In short: School is boring, but I’m still doing well.

One of my classmates has just recently rigged a tournament bracket for the class (competing with the tests). It’s just like a typical college basketball bracket, except that to advance you have to “cover the spread”, that way those of us who are more intelligent *cough* don’t automatically advance over the other contestant.

There is a flaw to the system though. Our class #1 and I figured out that if he were to place against one of the better Electrical Engineering majors in our class for the quarter finals (which is a EE test), that he would have to score better than perfect to advance due to the way they are currently calculating the spread. By the way, unless regrades go my way, I will have been eliminated from the first round by 0.02 points, thanks again to the spread. ;-)

In other news: I’ve been hanging out with some friends more on the weekends. But more important, my wife has been hanging out with one of my friend’s wives. This is awesome, we’ll get her to have someone on her phone list yet! ;-)

I’ve been spending a lot of time trying to overhaul the kdesvn-build documentation. It’s still not done yet but hopefully it would be much improved by the time I’m done.

One last thing: Jonathan, are you sure they didn’t just remove the logo for a different reason? If it’s true what you claim then I am very disappointed but I think it’s bad form to simply make an accusation when it could be a simple misunderstanding.

Posted by mpyne under Uncategorized | No Comments »

I’m just responding to a few posts I saw on Planet KDE.

First off, a KDE contributor has joined the military (U.S. Army) and was offering his thoughts. Since I have already joined the U. S. Navy I thought I’d offer some (public) advice.

First off, the Voltaire quote he paraphrased wasn’t actually said by Voltaire (although he said something similar in meaning). But it’s “public knowledge” now so don’t fault Aaron for that, I just wanted to clarify. See the Wikipedia entry for more information.

Also, although I suppose it may be different for enlisted members of the military (especially those that aren’t actually in yet), it’s not the best idea to be discussing pretty much anything political on a very public forum. Especially when discussions turn to the current Commander-in-Chief of the Armed Services. Of course, I’m not a lawyer, etc. etc. etc.

I would like to point out to those giving him grief that attacking members of the military does nothing to help your cause. It is the civilian leadership that holds power over little questions such as invading sovereign nations over possibly misleading or trumped-up evidence, or spreading Reserve and National Guard units around the world so that they won’t be available to provide assistance to their home states in times of crisis. I don’t know how it works in other countries but here in America the military is rather strictly under the control of the civilian leadership. This is a good thing, trust me, but this means that your average grunt is as powerless to change military policy as a civilian. The military will exist no matter what anyways, so would you rather have those who are driven by the desire to help and defend their country in the Army, or those who would rather spread destruction across the world?

I also wanted to reply to Tom Chance’s concerns over KDE naming. He says that if KDE adopts a term for applications that are shipped by KDE in addition to the base libraries for development, that 3^rd party developers would also adopt the term and render it useless. But you could apply this reasoning to any of a similar set of situations. I don’t want to go too far into it but it basically implies the kind of malicious intent that is always brought up when discussion sites like Wikipedia but which never seems to actually materialize.

Tom does make a good point that to use a term for KDE’s own applications, there should be a set of guidelines that such apps would have to follow, which would have to be public and have requirements that applications would have to meet. Tom brings this up as if it were an issue, but it’s actually a great idea, and as far as I know we’re already on the way to doing things of this nature. Those who read the kde-core-devel archives will notice that it is becoming harder and harder to say “Hey, I’ve got this great app which I think KDE should ship!”, as more and more is required of applications hoping to make it. I think it would be great to codify what we expect KDE’s applications to support as a minimum base for KDE 4, and either fix apps that aren’t up to spec or boot them out when we do the release.

Also, he’s concerned about the fact that there may be confusion when people say they have a KDE application as to whether they are talking about an application for KDE libraries, or an application by KDE the project. But, Aaron Seigo’s proposal has already taken care of the latter argument. Although I think a better name suggestion is in order, so I’ll throw out a few. First let me note that I don’t want a proposed name to start with a K. Tom already mentioned that if we used Kaleidos as the name, then we could refer to KDE as the Kaleidos Desktop Environment. But it’s not that, it’s just KDE, but why should people think different if we lump our applications under Kaleidos or any other name that begins with a K?

Personally I’d like such a name to be generic as well, and relatively short. Right now I’m leaning towards astronomical names like Terra or Neptune. But I’m not sure if either is used in the computer field. Oh well, I guess it’s good that I don’t get paid to think of names. :)

Note that I’m not unreceptive to Tom’s suggestion that we just leave things alone and pound in the meaning that a KDE application is one built by anyone using KDE libraries. But if this is a confusing issue to people, then Aaron’s idea of naming the kde.org application set makes a lot of sense, and I don’t think it will suffer from the issues that Tom posits.

Posted by mpyne under Uncategorized | 1 Comment »

Seb Ruiz had a question about whether unusable features should be removed from context menus or grayed out.

In his case he is inquiring about the ability to burn CDs from a playlist. It turns out that the feature he’s referring to was actually originally authored by myself as part of JuK. I’m not sure how it’s changed in amaroK since then, but in JuK the feature was always compiled in. It was only disabled if the CD-burning program was not found installed when JuK started. So it would make sense to leave the option there in the menu but disabled.

Turns out that the option is removed if it can’t be run though, in JuK’s case. Or to be more specific, the option is not added if it can’t be run, since the menu in question is constructed dynamically. Maybe I’ll change that someday since I actually like leaving the options there when possible and just disabling it if it can’t be run, if only to keep the menu from changing around.

Posted by mpyne under Uncategorized | No Comments »

Since there’s a documentation freeze (and it’s not like I’d, uh, get around to doing a great job updating them anyways. :-( ), I figured the next best thing I could do would be to explain somewhere what the deal is with the cover management code for the JuK release that will be part of KDE 3.5.

First off, a screenshot of one new feature: JuK will try to detect when you’re renaming files that have cover art, and if a folder has the album name, JuK will automatically set the folder icon to the cover art. :-)

Anyways, the new cover management code opens some new possibilities for users. In KDE 3.4, the cover for a track was strictly tied to its artist and album name. Although this proved useful enough, and had a few advantages, it wasn’t a great way to organize the covers. If you wanted to use a cover for a different track, you either had to rename the tags in the track, or you had to duplicate the image. And if your track had no Artist or Album information, JuK would prevent you from setting a cover since it had no information to go by. It worked, but it could be better.

In KDE 3.5, it is better. It’s still not perfect, and I’m not really happy with the Cover Manager dialog window. But it works, and it can save you time while allowing you to do more.

So just as an example, let’s say you wanted to set a cover for tracks you just ripped off of your CD. We’ll use Alabama - Greatest Hits III for the sake of discussion. In KDE 3.3, you could simply select any one of those tracks, and import a cover from the Internet by right-clicking, and using the Cover Manager -> Get Cover From Internet command. As a side effect of the way JuK worked, the cover would be immediately applied to all of the Alabama - Greatest Hits III tracks, whether you wanted that or not.

In KDE 3.4, the procedure is exactly the same, with one exception: You should select all of the tracks you want to apply the cover to first. So you would select all the Alabama - Greatest Hits III tracks before using the Get Cover From Internet command. Or if you only wanted to set cover art to half for some reason, you’d only select half the tracks. Don’t worry about duplicating covers, either: JuK is smart enough to re-use the same image, so you won’t get 14 duplicate .png images cluttering your hard drive.

But what happens if you forgot to select all the tracks you wanted to tag? You could select them and repeat the process, but that would leave a duplicate cover on your hard drive (JuK can’t quickly tell whether two images are identical, maybe I’ll do MD5 hashing later or something). But that’s OK, you can tell JuK to use the cover from another track.

There are two ways of doing this: The first is to open the Cover Manager dialog using the Tagger menu (Tagger -> Cover Manager > Show Cover Manager). The Cover Manager will display a list of all the covers JuK knows about on the right, and after they have loaded you can quickly pare the list down using the search line, or by using the list of Artists on the left. Once you see the cover you want to use, you can drag-and-drop the cover onto a track to apply it. It should happen nearly instantaneously since JuK is reusing the same image (and you’ll see the cover while you’re dragging it as well). Unfortunately, it can take awhile to load the covers, and the Cover Manager isn’t really useful for much else besides.

So this leads to the second method, the one I prefer to use because it’s rather easy. All you do is double-click on the track that has the cover you want in order to start it playing. This will cause its cover to show up in the Now Playing bar, and you can drag-and-drop the cover to the track you want to change exactly as you would for the Cover Manager.

Also note that you can use drag-and-drop to quickly apply covers to more than one track. Just select the tracks you want to apply a cover to, and drag the cover onto one of the selected tracks.

So, that’s about it for my introduction to the new cover management code. If you’re wondering what else it’s good for, here’s some more suggestions:

1. Applying the same cover to tracks with Albums that have (Disc 1), (Disc 2), etc, which you couldn’t do in 2.3 without duplicating the cover.
2. Applying a “generic” cover to tracks if you simply must have a cover on every track, or if you have music that wasn’t released as an album but fits a genre well, you could make yourself a cover for that type of music and apply it to the songs in question.

Posted by mpyne under Uncategorized | No Comments »

Aaron Seigo recently mentioned JuK as an example of why it is useful to have powerful application development frameworks in order to seed great applications.

Aaron said this: “we ship with juk, a great little media player that is playlist centric, kicks ass at tagging and otherwise tries its best to stay out of your way while playing your music. meanwhile, amaroK is an application that is developed by people in our greater development community which is so amazingly killer in its network integration, hardware player support (think: iPod) and general eye candy and usability sweetness that the Ubuntu project actually bent their freeze date rules to allow it into their next release! and instead of fearing for juk’s popularity, everyone (including the juk authors i know) cheer for this success.”

Well that’s certainly my view, and I know that Scott feels the same way too. We will work on JuK to continue to refine it and make it better, but meanwhile we’re just as happy when amaroK continues to improve. And indeed, you’ll find both my name and Scott’s name listed in the amaroK credits thanks to code contributions that the amaroK devs saw fit to use.

In a way, it makes things easier on us. There are many things that amaroK does that simply wouldn’t make sense in JuK. And thanks to the ease of application development with KDE, it’s not unusual for two programs to both have vibrant development teams behind them. I already know the normal argument (why have two applications when you only need one!) but the thing people sometimes don’t realize is that JuK and amaroK are different programs, with different philosophies and intents, which would probably both come out bad from a merger. This is unlike the unfortunate distinct between KEdit and KWrite, which are two programs that should be doing the same thing.

Anyways, on the JuK front, I’ve been trying to fix the low-hanging bugs over the past couple of days, so it’s looking like 3.4.3 will hopefully be the most refined JuK release in awhile. You can see the current set of bugfixes for 3.4.3 at this page.

To potential bug reporters reading this: There are a few crash bugs reported against JuK, which doesn’t horribly surprise me considering the hocus-pocus we perform with pointers. =D But if you encounter a crash, it would be great if you could find a way to reliably reproduce the crash. This helps me even more than a backtrace (although the backtrace would still be appreciated).

In other news, I’ve ported abakus to KDE 4.0 (or the shambles that we’re calling 4.0 ;-), but since the buttons are ginormously tall I think I’ll leave it alone for now. :-/

Posted by mpyne under Uncategorized | No Comments »

“If you want something done right, you have to do it yourself.” Re-discovered that today. The problem is that I’m no good at dry cleaning.

The problem is that our command has us dress up in our Summer Working Whites on Fridays. Which is fine, except that the uniforms are simply beautifully white, which is to say that they attract dirt/dust easily. So I put mine into the dry cleaners last Saturday. And they shocked me by informing me that it wouldn’t be ready until Thursday (today) by 1530 due to a sudden influx from a visiting Taiwan Navy vessel. Well, whatever.

So I go to pick them up today around 1600. Not there yet, will be there by 1730 *PROMISE*. Now, I’ve worked at a Navy Exchange in a previous life, so I can see where this is going. I also know that I’m rather powerless to fix it at this point but they promise that the dude is already on the way.

So I go back at 1750, and lo and behold, the guy had arrived but his van wasn’t able to fit all the uniforms and he would be back at 1830. *sigh*. So we’ll try this again in about an hour, otherwise I’ll have to buy another shirt (thankfully it needs no tailoring).

Posted by mpyne under Uncategorized | No Comments »

I was reviewing the comments on the recent article describing the migration of KDE to SCons/bksys, and came across an incredible amount of disinformation.

The comments against SCons were usually of the variety that a system would suddenly be unsupported, or complaining that SCons uses Python, requiring an extra dependency.

Allow me to destroy those complaints. :-)

1. Bu-bu-bu-but think of all those AmigaOS users!!

The thing to realize here is that even with all of the baggage currently attached to KDE by the current build system, it is simply not possible to build KDE on systems that are more than slightly different from BSD or Linux, without patching the source and the build system. Indeed, the work that has been done using SCons has indicated that SCons actually makes it easier both for ports to Windows and Mac OS X. In addition, there is no reason that 97% of KDE developers and builders should have to suffer just because of the supposed benefits of the autotools. Nowadays any benefits we may have received are vastly outweighed by the disadvantages, and SCons is not only a “good enough” replacement, it is simply better, flat-out.

It is literally just insane how much extra time and space the autotools requires to support systems that KDE doesn’t run on, when SCons/bksys is able to provide everything KDE actually does need. I can distribute a SCons/bksys application, which includes SCons and bksys, in easily one-sixth of the space of an autotooled application (that does not include anything but the configure script and the required /admin directory). For smaller projects, SCons/bksys will beat the total configure/make speed probably by a factor of two, and although the effects will be less significant for larger projects, they’ll still be noticeable.

2. Bu-bu-bu-but SCons uses Python, it’s evil!!

No, Python is not evil. I’ll admit that it’s certainly not my favorite language, as Python has far too many warts to make me comfortable. But, unlike what the popular opinion seems to be about the autotools, they do not rely on just a POSIX shell. They also require a macro processor called m4, and the Perl language to be installed. The fact that POSIX shell is even used means that the scripts run are very slow since unlike with Python, a .sh script can’t tell what bugs/features are in the shell it’s being run by without running a great deal of tests and workarounds. We don’t need that cruft. To top it all off, it still practically doesn’t work without GNU Bash, so all those tests for esoteric shells are for naught, and do nothing but slow us down. This is to say nothing of the Perl dependency brought in by automake, or the fact that it’s practically impossible to actually use autoconf and automake correctly, no matter how many docs you read. In fact, with the adoption of SCons/bksys, KDE has a net reduction in languages since developers don’t need to know Makefile.am syntax, make syntax, m4, and bash syntax just to get stuff done.

Another thing is that the autotools also require good ol’ make. You know, the dependency tracker straight out of the 1980’s? It has its place, but there is a reason that we replaced make with unsermake, and that is because it is increasingly out of date with the reality of build environments nowadays. make doesn’t really support distributed compilation well, and the effects of automake tend to cause make to run very slowly.

So in conclusion, this is an astoundingly awesome change. For those who simply can’t live without all the megabytes of cruft that the autotools entails, feel free to maintain a KDE build system yourself. :-P

Posted by mpyne under Uncategorized | 1 Comment »