substellar@Safe-mail.net Jul 17, 2010
> The paper is here http://arxiv.org/abs/1003.5852file ppmxl.dat
>
> The just released VizieR sampling access (the full thing is around 40 Gb) is here for the readme http://cdsarc.u-strasbg.fr/viz-bin/Cat?I/317 and here http://vizier.u-strasbg.fr/viz-bin/VizieR?-source=I/317 for the data. Around the end of this I'll outline how to use it in Guide 8.
>
> Now. The PPMXL is a main attempt to clean up USNO B1.0 which is not great on proper motion correction and is also full of spurious objects, which is not me saying that, the USNO people have published a paper themselves saying that. USNO B1.0 was a major first effort in things of that nature and thus they wanted to be as inclusive as possible, not missing anything, so at the faint end lots of nonexistant stars are included, and these faint objects can often be given large proper motions, when they've been wrongly linked to real faint stars in past epoch plates. Next time someone does such an exercise it will probably stress on accuracy rather than inclusivity.
>
> The astrometrists, especially the asteroid guys, can get very much like political factions, or sects of a faith, when it comes to using catalogues. Many will use NOMAD for instance when at the faint end they are still only using USNO B1.0 with no modification, but they believe and have faith that it is better. That is, there will be strong adherence to a manifesto or doctrine irrespective of information, but more on the belief placed into one catalogue or other.
>
> As outlined in the paper, PPMXL has deliberately aimed to solve some of the problems with astrometry in USNO B1.0, and bring it onto the ICRS. Some small discussion occurred about it some time ago on MPML, where some professionals seemed to think it should solve some problems, and some amateurs showed some interest in testing it.
>
> However, until today's VizieR release it was only accessible as very small downloads via VO or as the whole thing, and the latter not easily. Possibly it may appear in astrometry packages soon that bleed their data out of VizieR.
>
> Now, I've put a lot of effort into this TDF because there are important points in there, and although astrometrists will make great noises about accuracy and which catalogue is best, they'll often not bother using the errors in the catalogue or the quality flags, and using it as is, uncritically, declaring it "the word" in astrometric resource (else most of them would use a bunch of catalogues depending on what bit of sky they were working with instead of catch all general compilations not without some problems in them, like they usually use).
>
> I've laid out the extra bits of info this catalogue gives in Guide 8 More Info here. Areas can be browsed, and if you have all of the catalogues, UCAC3, PPMXL, USNO B1.0, NOMAD can all be compared to each other in Guide on screen.
>
> It should also be noted that there are still erroneous and false objects in PPMXL. Some devotees of particular other catalogues will rabidly fall upon that as an excuse not to assess it relative to other catalogues. Of course these errors also exist in NOMAD and also exist in USNO B1.0, because these false objects have come from USNO B1.0 in all cases. PPMXL explains it, it is an attempt to clean things up in USNO B1.0 using 2MASS as a reference anchor. Or, if preferred, to some extent it's an USNO B1.0 and 2MASS cross, which some people have been asking for for a long time.
>
> On to the data :-
>
> In PPMXL magnitudes have been taken from the USNO B1.0 plate sources, and details kept in the catalogue. So I've taken the route of downloading all columns from VizieR (VizieR makes things easier, as PPMXL comes in lots of zone datasets, whereas VizieR leaves that invisible to the user, merging them in a hidden way) and keeping all data.
>
> Some comments are included, such as how the position errors are derived, and some are only echoed when relevant to the object being investigated.
>
> For the brighter objects PPMXL has imported from the earlier PPMX http://vizier.u-strasbg.fr/viz-bin/VizieR?-source=I/312 which itself has imported from Tycho2 and an Hipparcos recalibration. So this is as complete as NOMAD, which is only a mix of all these catalogues (along with UCAC2). UCAC3 was not used in PPMXL because of the many problems with it, which again the USNO people admit openly. The difference between NOMAD and PPMX and ARIHIP and PPMXL is these latter are not just catalogues shoved together, some work on merging and calibrating data has been done, in the case of PPMXL using 2MASS as an indepedent to USNO reference. So, PPMXL will have objects too bright for an USNO B1.0 to 2MASS crossmatch imported from PPMX, which itself is seeded via Tycho2 and Hipparcos for stars missing from it, so technically few stars should be missing. Usage will show what is true.
>
> There is a bitwise flag in the VizieR served data which is 0 for no comment, 1 for bad chi-squared values on one of the positions, 2 for imported from PPMX, 4 for one object replaced with a PPMX object, and 8 for multiple objects replaced by a PPMX object (USNO B1.0 at times will for example have false stars surrounding bright stars, which are records of image defects on the original survey plates, not of stars).
>
> In practice such combinations usually lead to values like 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14 and 15 standing for combinations of the numbers 0, 1, 2, 4 and 8. That is, 3 is 1+2, which means the thing that is shown by value being 1 and the thing that is shown by value being 2 are _both_ true. Thus flags can be written as a number between hexadecimal 0 and F (0 to 15), and instead of having 4(???) columns in a table for "yes/no" situations you only need one. This makes things complicated in Guide TDFs though.
>
> However, the language logic shows that many of these are in fact redundant in the case of this particular catalogue. All the odd number ones do not apply, because the flag value 1 means an excessive chi-squared problem for one of the coordinate axes as measured via the USNO B1.0 to 2MASS match. This cannot apply to any imported PPMX data, so no other odd number than 1 applies.
>
> In practice, examining fields, it turned out that the objects flagged with a 1 were often peculiar objects, and what would occur is one PPMXL object would occur with a good solution, whilst immediately next to it would be an object with flag set to 1, and patently daft proper motion values at several 100 mas/y levels. Accordingly I have set the following up so as to _not_ download any data with flag set to 1.
>
> That leaves the even flag numbers. 0 is fine, that just means "nothing to note".
>
> 2 means imported from PPMX, 4 means one object from USNO B1.0 replaced by PPMX, 8 means several objects from USNO B1.0 replaced by ne PPMX object.
>
> So, essentially, 2, 4 and 8 mean "from PPMX".
>
> Strangely, for whatever reason, 8 appears not to be used. All values for multiple object replacemnat are set as 10 it seems, as 8 means replaced from PPMX, which means 2 (imported from PPMX) is also true, and 8+2 is 10. That would mean 6 would also be true, as 4+2 = 6, for the single objects replaced, and thus imported, from PPMX. But no, it seems there are no sixes. Only twos. That is, the 2 flag is used for that case, somewhat inconsistantly. For a test field of ten degrees around 0h RA and 0 Deg declination only flag values 0, 1, 2 and 10 were used.
>
> I've still included range testing in the TDF however just in case other fields do use the values 4, 6 or 8 values instead of the 2 and 10 values...
>
> Juggling all these somewhat contradictory and overlapping flags I've tried to set the constraints up so that the TDF works fine in all circumstances, and have tested a few mutually exclusive cases to ensure they only report the relevant data, well, relevantly. You see, the data columns represent different data depending on the source catalogue used, and only these bitwise flags advise where the data came from. For PPMX imported sources the five optical magnitudes are catalogue mag, Johnson B, UCAC red, not used and Johnson V, but for the PPMXL generated data the magnitudes are taking from USNO B1.0 (when they exist) and are B1, B2, R1, R2 and I. The tdf is setup to look after all this automatically without the user worrying about it, and display it as clearly as possible (however it was still impossible to stop null values being shown for some usno b1.0 magnitudes, rather than leaving that line out altogether, ie if there is no R2 magnitude in USNO B1.0 instead of no line being written, R2 followed by blank is written, as there's no way around this without ability to use complex combination conditionals, which is beyond basic conditional testing).
>
> Data given are the rather involved long number PPMXL identifier, full catalogue precision RA and Dec, converted on the fly to sexagesimal by Guide in More Info also quoted directly in full precision in decimal degrees from the catalogue, and the proper motion. Full details of the catalogue quoted errors are given in More info, and epochs etc, and whether one object is replacing multiple USNO B1.0 objects noted. USNO B1.0 magnitudes and which survey, emulsion plates, declination zone range, field, date range and wavelength range (lambda) are given if the magnitude data come from the USNO B1.0, or a line saying from PPMX if from PPMX.
>
> Basically, it's pretty much of the ilk of the Guide 8 USNO B1.0 tdf, carrying as much information as is given.
>
> The 2MASS J and Ks magnitudes are also displayed if they exist.
>
> A star to the magnitude size as based on B1 or MAG is plotted, but in order a) to show which these stars are relative to all other stars that might be plotted up by Guide, and b) when there is no B1 or mag value known (these two are usually the most commonly existing magnitudes, that's why they were picked as "representatives"), a large purple + symbol is also plotted centred on the star, or the star's position.
>
> 900 million objects can be now sampled from and plotted in Guide via VizieR.
>
> So, how to do it?
>
> Pre-first, make sure Guide is _not_ running.
>
> First, look for download.txt
>
> Go into your Guide 8 directory (it _must_ be Guide 8 or above because of some of the conditional testing done in the TDF needing that level of sophistication).
>
> Backup download.txt
>
> Load it into a text editor.
>
> Add this line :-
>
> 2805 -cI/317/sample&fl=0,>1&-out.all=1 ppmxl.dat
>
> into the file before the line that just has the word :-
>
> end
>
> save it as is, with no change in file extension, back into your guide directory.
>
> If you already have an entry number starting 2805, use the next available free number, and _remember_ it, you'll need it in toolbar.dat.
>
> If you want, for no good reason at all, to include the objects flagged as having absolutely spurious proper motions, then simply use this instead
>
> 2805 -cI/317/sample&-out.all=1 ppmxl.dat
>
> Now, backup toolbar.dat, also in the Guide directory.
>
> Load toolbar.dat into a text editor.
>
> Search for the word USNO to find where the line saying something like "usno b1.0 from VizieR is.
>
> Insert this line around there somewhere (where you place it will dictate what order it appears in in the toolbar relative to other buttons) :-
>
> 2805 !ppmxl.bmp PPMXL data from Vizier
>
> with no preceding #, and ensure the columns align with the other entries.
>
> Go to the top line and increment the number there by 1, as you've added one line, and this is the line count.
>
> ie if it says 0323 change it to 0324.
>
> Save it, without changing the name or extension in any way.
>
> Ah, but you haven't got a button for the toolbar...
>
> I'll put a file up on wikisend, see below, but you can draw your own either by copying a .bmp file already in you Guide directory and pixel editing it in a graphic's package and renaming it once finished to ppmxl.bmp and placing it in the Guide directory.
>
> Now, the TDF.
>
> -------------cut and paste into a text editor starting from the next line---------------
> ---------------the line previous to this one is the last line to be included in the cut and paste highlight-----------
>
> paste it in a text editor and save it in the Guide directory as ppmxl.tdf and exactly as that, no ppmxl.tdf.txt or anything else.
>
> One line might wordwrap
>
> ~r 1 1 \n\nPosition errors are sigma(J2000) = sqrt[sigmameanepoch**2 + ((2000-meanepoch)*{sigmapropermotion**2))]. RA error corrected for cos(Dec)\n\n
>
>
> that should be all one line with the first part aligned such
>
> ~r 1 1 \n\n
>
> with respect to spacing.
>
> You can delete this line altogether, it's always true and just a comment about the catalogue's errors.
>
> If you've rabidly decided to have all the data, even that flagged bad, then in the above this line
>
> ;~r199 = '1' One coordinate has an excessively large chi-squared value
>
> can be activated by deleting just the ; character at the front of the line. Then more info will tell you when an object is a bit problematic re accuracy.
>
> If you don't like purple plusses, remove them by deleting this line :-
>
> type sCff0080;+32;
>
> and you'll just get stars plotted (that is, for objects with a B1 or mag magnitude only).
>
> If you want the plusse, but not the stars delete the line saying
>
> type 6
>
> If you want labels, then activate labels for right click, then the DISPLAY button. You'll soon change your mind about wanting labels...
>
>
> Run Guide. Default is show at 1 degree or less, the DISPLAY button from right clicking on an object will allow you to change that.
>
> Of course, you've to be on the internet to access from vizier, but there we go.
>
> Data is downloaded into the Guide directory, cummulatively, into the file ppmxl.dat.
>
>
> And remember, any problems with PPMXL doesn't necessarily mean that it is worse than anything else, because the same problems of artefact stars will exist in USNO B1.0 and NOMAD, because the artefact stars all come from USNO B1.0, which the USNO NOFS staff admitted and stated in a refereed published paper.
>
> And these artefacts do still exist in PPMXL, it doesn't seem to have gotten rid of all of them Some dots still appear around bright stars, caused by UNSO B1.0 not dealing with overexposed stars properly, for instance.
>
> But ostensibly you should be far better catered for with respect to proper motion of faint stars, and even in some cases better catered for than when using UCAC3, with these data.
>
>
> However, the TDF is not fully complete, I've just remembered...
>
> For it to be fully complete it would take the proper motion values and correct them for "now" on the fly when plotting in Guide. I've never learnt how to do that bit.
>
> I do not use that feature in TDFs, and I do not know how to use it, so although this is usable as is if you want to look up values, and although most stars are at a very similar position in both 2000 and 2010, for the absolute purist they'll have to wait for Bill to have time to put that correction into the TDF, or into another TDF, and/or for him to include this catalogue download capability into Charon.
>
> Hopefully the somewhat lengthy outline here, and the column values of the vizier output etc, will make that task a tad less arduous.
>
> The zipped tdf and bmp button file (just put both in the guide directory) should live for about a month at
>
> http://wikisend.com/download/498534/ppmxl.zip
>
> Cheers
>
> John