Tag Archives: gstreamer

JuK Supports Opus Codec (sort of)

I just wanted to drop a quick note that JuK, the KDE music tagger/player component of the KDE Multimedia Software Compilation, now supports playback and limited metadata editing of Ogg Opus audio files.

This requires support from some underlying libraries:

  • The Taglib library must support Ogg Opus audio to add the track to your JuK collection at all. At this stage JuK can’t load a file to play if Taglib doesn’t support it, even if the playback system (Phonon) would otherwise support it. Taglib also provides the metadata reading/editing so our support is only as good as that provided by Taglib. For instance, the bitrates all show as 0kbps for my current encodes. Taglib only supports Opus in git as far as I can tell, so JuK uses configuration checks to see if Taglib supports Opus. JuK will still compile if Taglib doesn’t have Opus support.
  • To actually playback Opus audio, Phonon must support playback. Practically speaking this means your Phonon backend (VLC, or gstreamer, normally) must support Opus playback. Both underlying libraries already have releases (VLC 2.0.4 or later; gst-plugins-bad) supporting Opus so this shouldn’t be too hard.

One caveat is that I’m not sure what mimetype will eventually end up being used in shared-mime-data for Ogg Opus audio (probably audio/x-opus+ogg or audio/opus+ogg). At this point Ogg Opus audio gets detected as audio/ogg so I simply check for .opus extension if this fallback mimetype is identified.

In case it’s not clear from my writing “Ogg Opus” everywhere instead of just Opus, JuK currently does not support Opus audio in other container formats (such as Matroska). However the Opus standard recommends using Ogg as the container for purposes of storing audio as files so this should be how you encounter Opus files in practice.

Retro tunes with Phonon

So, in The Beginning, when I was just a young padawan on the Internet, I had been let into a glorious secret: Emulation (not of IBM System/360 machines, but of more important things like the Super NES). Some branching from there led me to zophar.net, a popular emulation site, and their message boards, and also left me with a fascination with emulation.

The attributes of some of the older systems like the NES and Super NES made it fairly easy to capture their music-producing software, since those systems used separate co-processors to handle music effects. NES music would be stored in the NSF format, and SNES music was handled with the SPC format (named after the audio chip used, the Sony SPC700). There were (and still are) specialized plugins on many systems to play these formats (they emulated only the music chip, not the rest of the system).

I’ve been involved on the periphery of some of these things for the past couple of years. (For instance I had written a KFileMetaInfo plugin for KDE 3, and had helped Chris Lee with adding playback support to GStreamer.

One problem with the previous GStreamer solution (which I’ll call gst-spc) is that the underlying playback library, libopenspc, is written in x86 assembly, and has some crash bugs associated with it as well. As well the code has long been orphaned. I’m not really any good at writing emulation code and although I could learn, it would take far too much time for me to do anything useful.

Luckily for me the state of the art has advanced and last year I was pointed to a library called game-music-emu. This library included a very good SPC emulator written in C++, which had been merged into some popular SNES emulators already. Unfortunately it didn’t really have a great build system (using it involved simply copying it into your existing program) so my initial proposal to port GStreamer to use game-music-emu by simply including the source files with GStreamer was rejected. The GStreamer devs preferred to have an external library which could be used (or not) and I couldn’t blame them since in general good OSS projects avoid copying or forking external code.

So I contacted the game-music-emu author (Blargg) asking about the possibility of adding support for building a library, and ended up with commit access and an invitation to do it myself. Hmm.

So I did, and awhile ago I had made a release of “libgme” 0.5.5, working with Blargg has he got free time. My subsequent patch to GStreamer was accepted and since gst-plugins-bad-0.10.14 it has been possible to use libgme to playback many emulated music file types (not just SNES, but others as well).

With that solved I left the issue, but I recently came back to it since I figured out that even after upgrading to gst-plugins-bad-0.10.17 the other day, that gstreamer playback was not using libgme, but the older libopenspc.

At first I thought it was simply my fault, as I’d still had gst-spc installed from years and years ago. Removing gst-spc and libopenspc (just to be double-sure) left me with no SPC playback features. Running gst-inspect confirmed I did not have any gme decoder. WTF.

I then again thought it was my fault because I had installed libgme to /usr/local instead of /usr. So I dutifully wrapped up libgme in an ebuild and installed it. And still nothing. WTF.

I dug into the Gentoo ebuild for gst-plugins-bad and it seems that for whatever reason not all possible plugins are installed. It seems the new installation method is supposed to be that each individual plugin is supposed to have its own ebuild (i.e. gst-plugins-gme), like how Gentoo has split out other packages like KDE into individual ebuilds. Fair enough.

I write another ebuild, and finally hit paydirt:

Screenshot of music player playing SPC files
The Qt example music player playing SPC files

Obviously this does require that you are using the GStreamer backend for Phonon to have this work, otherwise you can just try it in some other GStreamer-using application. (I’d show it in JuK but I’d have to add SPC support to Taglib first)

If you’re interested in the ebuilds I used you can use this Portage overlay, (SHA-512 sum c0ff9aa5413b0c0b14f7c52d5b3ee887edc4e7bf47182e58c21e9c340d8ff7e9). The overlay may or may not work for you, and I don’t even know if overlays are still the “hip” way to do things in Gentoo, but It Works For Me. ;)

Status update

Since I haven’t blogged in awhile I thought I’d give an update as to what I’ve been doing in the past month or so:

  • This is only tangentially related to KDE at best but I’ve been pushing to get an improved video game music emulation library supported by GStreamer. The library in question is simply called Game Music Emu (or libgme depending on where you’re looking ;). It is an all-in-one emulation framework allowing for decoding and playback of Super NES, NES, Sega MegaDrive/Genesis formats and more. This has ended up with me having commit access to libgme and fostering a mini-revival by the library author to turn it into a proper library. Based on this work, the GStreamer devs have applied my patch to use libgme and then improved my patch several times from there. The next releases of gst-plugins-bad (for SNES SPC, etc. playback) and gst-plugins-base (for the typeinfo fixes) will be able to make use of these changes. JuK requires TagLib support to add files to a collection so even if you use phonon-gst you’ll still need to use a separate player to test it out though (Qt’s example musicplayer is perfectly sufficient though).
  • I haven’t forgotten about kdesvn-build’s git support (it’s actually there, but not plugged into anything other than qt-copy). My major hiccup has been handling the case where the user changes the “repository” option on me (especially with regards to qt-copy). I may just punt and make the user manually do it since I’m not sure what the best way is to switch over the remote tracking options in git (i.e. make git pull work from the new repository from now on)
  • I’m trying to get started in a Master’s degree program for a M.S. in Computer Science. I’ve taken the required entrance exam (the GRE) and although I’ll not post the exact scores I will say I did well (and without studying to boot. I tried to study but couldn’t get the “PowerPrep” software to work in Wine). It’s been a bit of a special case for me as I was a couple of weeks past the deadline due to the timing of showing up at my present command, but I think everything will work out to start ASAP.
  • If you’re just getting started with KDE 4.3 and you start seeing dialogs warning about being about to start executing a file, that’s by design. I’ve heard of a bug where dragging a working desktop link will make an “unsafe” desktop link since the destination doesn’t fall under the same exemptions as the old location which I may try to look into. Just remember that this is for your own good, and is a one-time only dialog per file! If you are writing your own .desktop links which you want to launch applications, just make sure to set it as executable.
  • Finally I’m going to be putting an old unused computer of mine to good use and setting it up for my 2.5 year old son. He enjoys computers too much, now it’s time for him to click on his instead of mine and my wife’s! ;) So I’d prefer it to have a guest account arrangement on super-lockdown (no Web, edu games, paint applets, etc. available, locked or healing desktop, that kind of thing). Any suggestions for KDE-friendly distributions for this kind of thing? I’m trying to keep download size for the install media under 1 GB.

Quick tips

I’ve rounded up some useful tips, none of which are really important enough to warrant a post just by themselves. So without further ado…

  • If you’re running kdesvn-build, you can use the --refresh-build option to force the given modules to be built with a clean build directory. But what if you didn’t want to do that for all of the modules on the command line? What you can do instead is create a file called .refresh-me in the toplevel of the module’s build directory (e.g. build/kdelibs/.refresh-me). When kdesvn-build rebuilds the module, if it finds that file it will perform the build process as if –refresh-build had been passed. Since –refresh-build involves deleting the build directory this is a one-time-only event. Next time you build everything will work normally.
  • Konqueror deserves several posts on its unique gems just on its own. But one thing I’ll mention is the address bar. There are two different keystrokes that I know of to quickly select the address bar: CtrlAlt - O will select the address in the address bar, handy if you just need to make a quick change to go to a different address. Ctrl - L on the other hand, will delete the text in the address bar first. I think this is a holdover from earlier days to be honest though, as since the URL is selected by default I can’t think of anything that Ctrl - L can do that is easier than using CtrlAlt - O. Update 2:One of the commenters pointed out that the selection buffer gets overwritten using Alt – O but is maintained with Ctrl – L. (i.e., when you middle-mouse-click to paste).
  • Do you use QDataStream? If so then it is imperative that you set the version of the stream. What I mean by this is that sometimes the binary representation changes depending on what version of Qt you are using. So a stream saved by a older version of Qt may be unable to be read using a newer version of Qt with the default version settings. QDataStream is backwards compatible; you just need to know what version to set the stream to. The process is described in the API documentation. Basically you need to set a specific version before writing and before reading. Some types (such as the integer types) are guaranteed not to change their representation, which you can use for versioning.
  • If you are a Qt programmer, then Thiago’s description of QString behavior is required reading.
  • Many people know that the C++ delete operator can safely be called on 0. Per the standard, this is also true for the delete[] operator, so you don’t need to check for null either way. (Of course, there are reports of compilers that screw it up for delete[], sometimes you can’t have your cake and eat it too).
  • And on that note, C++ has an interesting behavior that makes some variadic functions harder to use safely. To wit:
    • C has a NULL macro to represent the null pointer. Typically it would be along the lines of #define NULL ((void*) 0) or thereabouts. void * can be converted to any other pointer type so this is valid.
    • In C++ however, you can’t convert pointer types all willy-nilly. So the #define NULL ((void*) 0) doesn’t work. Instead you use plain 0, which C++ guarantees will be converted to the null pointer when used in a pointer context.
    • The key word here is “used in a pointer context”. Some variadic functions assume that the last parameter will be of a so-called sentinel value. It is up to the programmer to ensure they pass the sentinel correctly, and with the right type. For instance, many gstreamer varargs functions assume a null pointer will be the final argument.
    • If you use the NULL keyword things will work great in C and C++… if you’re on a system where int and pointers have the same size. On 64-bit systems this is not necessarily true.
    • The problem is that NULL is defined to plain 0 for C++. When 0 is passed as the last argument it is assumed to be of int type, not of a pointer type. For most 64-bit systems this will involve placing only 32-bits of 0 on the stack instead of the required 64-bits, and “undefined behavior” results.
    • The solution I used when I encountered this problem in KDE 3’s JuK was to simply define a GST_NULL that did the right thing. Other options are manually casting to a pointer type every time you use one of the vararg functions. But it’s something to keep in mind anytime you’re using variadic functions.
  • The upcoming revision to the C++ standard will include an actual null pointer instead of faking it with 0 and at least this kind of issue can be fixed. Some compilers have already included such a feature (including GCC) and so you may have never encountered this problem.

That’s enough food for thought for now though, hopefully this is enlightening reading for most. Update: The keyboard shortcut is Alt – O, not Ctrl – O.

Debate time

Why are you taking notes on the debate? I thought you already mailed in your vote.

This was a quote from my wife a few hours ago as the Presidential debate was getting started. She was right of course in that it wouldn’t affect my vote. I did jot notes down for the first hour or so just so I’d be able to remember the debate later should discussion ever come up. :-)

Anyways I expected better from Obama and worse from McCain. I still think Obama won but he at least at first seemed to have trouble collection his thoughts at times. I’m quite pissed at McCain trying to say that the only way for the deaths of American servicemembers in Iraq to have meaning is to “win” the war as it discounts the very real effort of many of those servicemembers. The Navy Times had an article the other day describing a Marine put up for a Medal of Honor because he died jumping on a grenade to save nearby Marines. He may not have won the war but you’d better believe what he did meant something to those Marines that he saved.

If there’s any single thing about this campaign that scares me though, it is Gov. Palin potentially becoming President if McCain wins the election. I’m impressed enough that he is doing as well as he is doing at 72 (?) years old but he has melanoma, suffered for years as a POW in Vietnam and, judging by the change in appearance of Presidents Clinton and Bush over their time in office, I’m really not convinced that Palin wouldn’t end up as President sooner rather than later. I feel that would be disastrous for the nation given her current practices for picking executive staff (i.e. nepotism instead of meritocracy). Being completely silent on the investigations going on in Alaska isn’t making me feel any better what with having just gone through Attorney General Alberto Gonzalez…

However, that’s neither here nor there and all I can do now is hope that the best candidates win the upcoming election. In more exciting news, a friend of mine coerced me into joining a fantasy football team (for points, not money, don’t worry). I didn’t really put a lot of effort into the picks and I missed the draft due to other committments so my team (“Graphitar”) is basically the best team the Yahoo! Random Number Generator could provide. Against all odds though, the team put up an incredible performance in Week 1 thanks to career days by Donovan McNabb and Michael Turner. It’s looking like the 171 points or so put up in that game may stay a season record at this rate. 6/8 teams get to the playoffs so I should at least be good for that.

On a completely unrelated topic I see the topic of Phonon and gstreamer has come up again. I documented many of the reasons why KDE needs Phonon in a letter to the LWN Editors more than 2 years ago. Suffice to say I don’t feel like going over the arguments again, but let’s just say that Phonon keep the doors open for gstreamer 0.12 to be supported in KDE when the time comes.

Finally on a more depressing note a submarine sailor was killed almost a week ago. The linked article is about all the specifics I have except that it is unlikely that the sailor was cleaning the rudder since last time I checked it had to be in the water (where the crew is not located) to work. It is likely that any cleaning he was doing was in the area of the hydraulics controlling the rudder but I don’t see how he would have been caught up in the mechanism. At least on my boat it has a shield surrounding the mechanism although Murphy’s law being what it is…

Improved crossfading

I’ve been improving JuK a lot over the past week or so trying to get it into better shape for KDE 4.1 (and 4.0.5).

What I’ve worked on today was the issue with crossfading where the volume differential between the songs would go haywire initially. I think I’ve got it as good as I can get it from JuK, but now there’s something else I need.

I have posted an email to kde-multimedia detailing my problem, but basically I want to use the gstreamer phonon backend to test whether the problem lies with JuK or with phonon-xine.

The first problem was that JuK wouldn’t work with phonon-gst because JuK made some inaccurate assumptions about when Phonon would actually start playing after being told. (Hint: Not immediately, the backend has to setup first).

That is fixed in both the 4.0 and trunk branches now. But when using phonon-gst to playback a song, any time I switch to the next track JuK crashes on an assertion in phonon-gst. This happens even if I do not allow crossfading. This does *not* happen if I slow it down in gdb by single stepping. So I think it’s a timing issue.

Anyone out there use phonon-gst and could tell me what I’m doing wrong?