DESI EDR tilecumulative vs. pix

Question

Dear DESI Helpdesk,

Which of the two files from DESI EDR should I use if all I need are good, unique redshifts of galaxies?

zall-pix-edr-vac.fits

or
zall-tilecumulative-edr-vac.fits

?

It seems to me that the former is a subset of the latter. Still, some TARGETIDs in the former appear more than once (some are repeated up to 5x). So my follow-up question would be, how to clean these up (I can ask it separately if needed)?

Best regards
Maciej Bilicki

Answer 1

Hello Maciej--

The "pix" catalogs are based on the healpix coadds whereas the "tilecumulative" catalogs are based on the tile coadds. A tile is a single DESI pointing of 5000 fibers, and since the same target can be observed on different tiles, you should use the "pix" catalog, which takes these cross-tile observations into account (within a given "survey" and "program").

These ideas are documented here--
  https://data.desi.lbl.gov/doc/organization/#tile-based
  https://data.desi.lbl.gov/doc/organization/#healpixel-based
  https://data.desi.lbl.gov/doc/glossary/#survey
  https://data.desi.lbl.gov/doc/glossary/#program

Now, to answer your question, I recommend using the "Redshift Value-Added Catalog", which corrects a few errors in the redshift catalogs which were produced when the EDR (Fuji) processing was finalized--
  https://data.desi.lbl.gov/doc/releases/edr/vac/zcat/

Specifically, I recommend reading--
  https://data.desi.lbl.gov/public/edr/vac/edr/zcat/fuji/v1.0/zall-pix-edr-vac.fits

Finally, this catalog still includes about 4% duplicate observations because we only healpix-coadd observations within a given survey and program. However, the ZCAT_PRIMARY flag will return the "best" spectrum for each unique target. So, in python:

import fitsio
cat = fitsio.read('/path/to/edr/vac/edr/zcat/fuji/v1.0/zall-pix-edr-vac.fits')
cat = cat[cat['ZCAT_PRIMARY']]

Hope this helps and good luck!

-John

Comments

Dear John,
Thanks you very much for this comprehensive answer! I've been using the pix vac catalog indeed.
Could you only clarify what the difference is between SV_PRIMARY and ZCAT_PRIMARY? The former flag seems to also return unique redshifts.
Best wishes
Maciej

---

It looks like 'SV_PRIMARY' only considers repeat observations within any of the Survey Validation (SV) surveys (sv1, sv2, sv3), whereas 'ZCAT_PRIMARY' is for any observation.

There's some documentation here, but we should probably make these ideas a bit more prominent:

https://desidatamodel.readthedocs.io/en/latest/DESI_SPECTRO_REDUX/SPECPROD/zcatalog/zall-pix-SPECPROD.html

https://desidatamodel.readthedocs.io/en/latest/DESI_SPECTRO_REDUX/SPECPROD/zcatalog/zpix-SURVEY-PROGRAM.html#hdu1

Kindly,
-John