PXML validation tool for Pandora apps
December 1, 2009
Developers, developers, developers, lend us your attention spans. By now you should be well familiar with the concept of Pandora’s .PND packaging system, which automatically populates the menu with icons when an SD is inserted. To play a part in this sciencemagic, your app must include a valid PXML file. And to make a valid PXML file, you’ll need to read and understand the PXML specification in full – then not mess it up. We live in the real world though, where not everyone reads specs, or they just make mistakes. Sensing a compatibility nightmare on the horizon, defacto PXML ambassador Dflemstr has cooked up a handy little validation tool to help keep the ants out of your .PNDs. If you’re a prospective Pandora developer, you’ll need not one, not two, but all three of these links:
Forum discussion | The PXML spec in PDF form | Dflemstr’s PXML validation tool
XML is a markup language, like HTML. Using XML to represent data structures (rather than documents) is a very bad idea. Please use something like Debian’s package lists format instead. The same as email headers format, on each line, key: value, possibly several objects per file separated by a blank line. This has the advantage that unlike XML bogosity it is actually readable and writable by humans with minimal pain and anguish. I haven’t looked at this particular bogus XML format I am just assuming that like every other XML-based data structure format it will be horrible. Sorry to be discouraging! Plain simple XML is very good for markup and terrible for data structures. End of rant.
As far as XML stuff goes; it’s not too bad, and it has an example file which you can copy and paste into your own final file.
Relax man, take a chill pill.
There’s nothing wrong with using XML for data structures that isn’t also wrong with using XML for documents. I could go so far as to say that XML is entirely suitable to represent particularly intricate data structures; moreso than JSON and certainly moreso than simple key:value pairs.
The PXML spec in question (pretty well defined IMO) /could/ probably be represented using a simpler format, but the use of XML allows for a more elegant (IMO) eXtensibility into the future.
And human-readability of XML documents is a non-issue. Being *developer*-read/writeable is what counts!
I hate XML as much as the next guy, but I found this article on the subject quite enlightening:
“All markup ends up looking like XML”
http://www.megginson.com/blogs/quoderat/?p=153
RK, sadly every point you made is wrong. Marked-up documents are quite different from structured data / record sets. Everyone who uses XML for purposes other than markup is demonstrating that they don’t have a clue about computing (or they are being forced by marketroids who still think XML is cool).
If the Debian control format is good enough for Debian, and the email/http headers format is good enough for email and http, something similar would surely have been good enough for Pandora packages, with the advantage (over XML) that it would actually be readable and writable. Most developers *are* human, in case you didn’t know that, and most don’t enjoy wading knee-deep through mires of XML.
If you think XML is more suitable than “simple key:value pairs” I think you need to get a clue.
At some point I will post a patch to get rid of the XML bogosity, not that I expect anyone but me will use it.
For the person who mentioned XSLT, wash your mouth out! A greater abomination than XSLT has not been seen in the history of computers.
@Sam Watkins
What are you prattling on about, PXML not being readable? Have you actually looked at the PXML specification, it’s a piece of piss to understand. Go read this page: http://pandorawiki.org/PXML_specification If you still don’t understand how its laid out then maybe start learning HTML first to get a feel for markup languages.
Package metadata is NOT the same as a marked-up document, which is why I am protesting the (ab)use of XML for it. I knew without looking what it would be like. Look at the example on that page you referenced, and let’s fix it…
Oh look, 575 bytes of readable text instead of 1154 bytes of ugly XML:
#!/usr/bin/prun
id: youruniqueID
title: en:Program Title
title: de:German Program Title
background: true
startdir: /usr/share/icons
standalone: true
command: program
icon: program.png
description: en:This is the English Description of the file.
description: de:This would be the German description.
preview: preview/pic1.jpg
preview: preview/pic2.jpg
author: Some Dudeson
website: http://a.bc.de
version: 1.1.1.2
osversion: 1.0.0.0
category: Game/StrategyGame
category: Graphics/ImageProcessing
association: View Word File; application/msword; -f %s -t doc
clockspeed: 600
I have seen how people in the education industry have abused XML to create such monsters as “SCORM” for describing education resources. I believe they are guilty of deliberate obfuscation, however they might just be plain stupid or ignorant.
I’m not saying the people who created PXML are stupid. Obviously you are not stupid; but perhaps it didn’t occur to you that XML is the wrong tool for the job. Perhaps you are accustomed to the abuse of XML in the industry for representing data structures.
All of the work on PXML could be saved if we switched to a sensible format like I have shown above. It’s only the abuse of XML that I object to, not the details of PXML.
Use XML for writing documents and webpages, and use something else for metadata and data records.
Perhaps it is not too late to change this, and thereby remove one likely cause of embarassment?
Even the freedesktop.org .desktop files do not use XML. They use simple key-value pairs, like redhat packages, like debian packages, like every sensible system under the sun.
@Sam Watkins
Your example would do the same job as the example on the PXML spec, I personally don’t think it is an improvement in terms of readability but then I like markup languages and find PXML perfectly readable, you clearly do not. I would argue anyone who has done a fair amount in a markup language (HTML, XML, etc…) before would be comfortable with reading PXML files, I’m a noob and I still found it easy.
Also, PXML is far from being an abuse of XML. Markup languages are designed to describe the structure of something, it doesn’t matter if that is the structure of a document or the structure of a data type. You don’t like XML, fair enough, but to say PXML somehow abuses the true use of XML strikes me as foolish.
Use of XML also allows for further improvements to the specification without it becoming a mess. For example, it might be decided to expand what is allowed to be used as a preview picture, maybe preview videos. Adding this to PXML would be as simple as going from:
to:
Adding video previews using your example would require that the parser knew to look for an alternative image and which has precedence, which is possible, but the more you add the more clunky your standard gets.
Does PXML start making sense to you now?
The tags in my previewpic example were hidden by this blog, basically you can change from “img src=…” to “vid src=… player=…” and still keep using the previewpic tags.
I also would prefer simple key:value pairs. It’s the best for flat data structures like those the pxml file implements. There’s no need for xml.
You can even do things like
category/game: StrategyGame
category/graphics: ImageProcessing
Or:
preview: foo.jpg
preview: foo.avi
No need for a file specification.
Not even parsing via software is cheaper this way. It’s more read- and understandable by humans. No need to remember complex xml structures (was it or ? Whats the case with preview pics? or maybe ?
There’s a need for xml, yes, but in case of pxml?… no.
Totally agree with Sam!
You can use XSLT to make the XML file readable. Lots of programs (and new programmers) know XML, so I guess it has some advantages.
I completely agree xml is shit for a configuration system old ini files can do 99.9% of everything in xml but doesn’t require nearly as much structure. Why make something complex when it can be simple.
err by that I mean the first poster… blond.
I think that we should be forced to write Perl code that outputs a properly-formatted PND when run
I have an Aunty Pearl. I could ask her to write some.
the libpnd distribution includes sh-scripts for spitting out PXML.xml, making directories into pnd-files, and all the stuff you woudl think you need. And peopel are working on GUI tools
I’m using XML for every aspect of my game (data, level,, object, physics, setup, etc.)
Really easy and modulable.
I was a bit afraid at first but in the end it’s a win.
Most of standardised program use this format and web is full of XML too.
Then will come MXML PXML XXML and shit like that to be as lost as possible
It’s like new OS afterall
…until you need to binarize everything to gain proper performance in your game
No need to binarize stuff for that project currently.
But I do have a support for serialisation, so shall not be a problem.
My game is smooth on a 500Mhz 128Mb ram P3 with 4 players full physics, huge level with lots of object, 4 tiles Layers, AI and particles effects. I think that’s ok for now
Hey Gruso do I have to read all these as well? You’ll be setting homework next.
I like this idea.
Developers may I have your attention?
http://www.youtube.com/watch?v=KMU0tzLwhbE
Unfortunately for those who aren’t keen on PXML, the opportunity to argue it passed about a year ago.
A lot of thought has gone into doing it this way (by people who really do know what they’re doing), because there were no other standards that fit Pandora perfectly. If you do a thread title search for PXML in the forums you’ll find lots of good discussion. Or if we’re lucky, Skeezix might drop by and defend his baby. 
But after all ? Isn’t the Pandora Open enough for anyone to try anything ?
Will be perfect I say.
In fact the only missing stuff right now is the Pandora itself.
In worst cases they can always send me all the pieces, I will then rent a small Chinese to build it.
ED made up PXML.xml aaaaages ago (18months?) Then we came along and debated it endlessly (tossing it out altogether etc) and in the end settled on keeping it, and refining it a bit. Then dflemstr came along and helped refine it and redo some of the parser and voila, its where it is.
Nothing is perfect; it works, and its light. ITs not some insane XML spec. ITs possibly to run through a validator and is pretty easy, and we have it working. Good enough
Anyone who doesn’t like it can easily alter libpnd to swap out PXML.xml support and use something else; they’d only have to edit maybe 2 files to stitch it out (Id’ build in an option for multiple formats day one, but I’m smart enough to know when to leave things out, rather than build too much in. Simple means less ‘surface’ to maintain and find bugs in.)
(I mean, we’re doing this in spare time, so cut us some slack
It is open however; source is up, ans has been since day one. People who bitch now missed their boat. they could’ve bitched a year ago when we got pulled in
Further, they can check out libpnd git, hack it up, and use that in their private dist.. but for shipping, we’re going with more or less whaty we have, since its pretty stable.
If people ant huge changes _now_, they can be my guest and tell everyone they woudl like to delay all the other thousands of folks shipments because of their personal goal
So as always .. we did a not bad job; ou don’t have to like it, thats okay, it doesn’t hurt our feelings
I’ve been coding a long time, you can criticize it all you like. I’ll be first to list off the things I don’t like about it myself, but I’m still shipping it 
There is a LOL face in the lower right of the picture!!!!!!
lol its the “awesome” face popular on ED and 4chan
Both are fine Bikeshedders.
I’ll stop bitching now. everything about this PXML format seems good (except for the XML). so that is a complimentary comment from me for a change.
Hey, I’m not too happy about the XML either (nor about the actual fields it contains! But I digress), but this issue was discussed at length (I think we went up to 30+ pages of forum posts), and we came to the conclusion that:
- The PXML file must be appendable to a PND, so it has to have a “header” and an “eof-indicator” that are easily recognized in the binary for a stream reader to find, so that we can use Knuth–Morris–Pratt to scan the file quickly (if you wonder, yes I think that the PND format is horrible too). In this case, the header is “<PXML" and the eof-indicator is "” as you can see in the libpnd source code.
- People who are on Windows must be able to write PXML files, so we mustn’t create a format that is required to be UTF-8 or end in “\n” instead of “\n\r” (or “\r” for that matter, for Macs)
- It has to be possible to open the PXML file without any external tools or some parsing spec, and almost every programming language/platform combination has an XML parser in some shape or form available for it.
Considering these issues, XML was the choice at the time. “.desktop” files were also brought up (because a PXML file is more like a .desktop file, and NOT like a package header) but for some reason we didn’t choose .desktop, probably because it conflicted too much with the above 3 issues.
Correction since WordPress removes HTML-ish tags from comments:
“and the eof-indicator is </PXML> as you can see in the libpnd source code.”
Escape that quote as XML to get what I mean.
As to “pnd format being horrible”; no one has yet suggested something _better_, so there you go
There are lots of _alternate_ suggestions, but we also tried a half dozen variations and found them all sub-optimal in the conditions; in an ideal world on a modern PC with lots of ram and ghz, its not so much an issue; but when trying to get a stable kernel with all the modules we need working it limits which drivers we can toss in; when mounting off SD then cramfs and zip and so on become performance issues, etc and so on.
For the folks who just like to complain .. you get yourself a beagleboard, and then make up 10,000 (yes, 10-thousand) sample pnd-files iny our favoutite haked up format, and see how well it performs to scan them all
Again, our format is not ideal…. but what is?
It works, and not half bad 
jeff
Democracy is the worst form of government except for all those others that have been tried.
- Winston Churchill
Also, once we launch, I’m sure we’ll all pooped and ready to step back; we’ve been focusing on getting somethign to launch, in the face of peopel wanting everything changed every other week, in the face of peopel wanting to build video codecs into libpnd (!), etc and so on.
Once we’re launched, the community can take over and go willy nilly; that’ll lead to a lot of chaos, and ultimately to some great stuff; more eyeballs and hands will tighten things up, make for better code and standards etc.
At that point, those who complain can run the ship; fine by us
jeff
By-cons, we should not wait too much longer.
Or you may be disappointed because of the pending birth of great expectations.
And I’ve always paid the price of this fact.
Now for the moment I found everything showed here encouraging and fine (except maybe the ugly silkscreen, were my vote and proposition didn’t count at all).
Hope they get a final version of the product soon because the N900 hit the ground and it’s really a nice little device.
Are version numbers designed to be used intelligently in the future (e.g. comparing numbers), or are they just display items? If the former, a quick note on how they should be used would be good. The thing that’s not clear to be is what the ‘release’ value is for: does the sequence go:
ma:1 mi:0 re:1 b:1 (for major, minor, release, build)
ma:1 mi:0 re:1 b:2
ma:1 mi:1 re:2 b:3
i.e. the release number goes up every time the app goes out of the door
Or is the release number actually more significant than the major version?
ma:1 mi:0 re:1 b:1
ma:1 ni:1 re:1 b:2
ma:2 mi:0 re:1 b:3
ma:3 mi:0 re:2 b:4
You can use any convention that you want, nothing is enforced. Just think about that you should have increasing versions e.g. after “1.0.0.0″ you should release “2.0.0.0″ or “1.1.0.0″ or “1.0.1.0″ or “1.0.0.1″ or “1.2.3.4″ but not “0.0.0.1″. That’s all you have to care about, ever.
i like the “awesome” face in the right bottom corner XD
Glad a couple of people noticed. It’s how I felt about dflemstr’s efforts.