[IGS-DCWG-34] RE: [IGS-DCWG-29] Re: [IGS-DCWG-27] Any DCs accepting .gz RINEX?
Schmidt, Michael
MSchmidt at NRCan.gc.ca
Thu Nov 18 15:01:14 PST 2004
******************************************************************************
IGS-DCWG Mail 18 Nov 15:11:33 PST 2004 Message Number 34
******************************************************************************
Author: Michael Schmidt
Hi everybody
To partially answer Carey's question of .Z vs. .gz vs. .zip files, here is a
brief comparison of compression of a 1 hr, 1Hz file (on Solaris):
-rw-r--r-- 1 wcdaop gpsusers 933624 Nov 18 21:49 bcdl320a.04d
-rw-r--r-- 1 wcdaop gpsusers 297759 Nov 18 21:49 bcdl320a.04d.gz
-rw-r--r-- 1 wcdaop gpsusers 291365 Nov 18 21:48 bcdl320a.04d.Z
-rw-r--r-- 1 wcdaop gpsusers 297884 Nov 18 22:11 bcdl320a.04d.zip
-rw-r--r-- 1 wcdaop gpsusers 2583764 Nov 17 17:55 bcdl320a.04o
-rw-r--r-- 1 wcdaop gpsusers 898794 Nov 17 17:55 bcdl320a.04o.gz
-rw-r--r-- 1 wcdaop gpsusers 961869 Nov 17 17:55 bcdl320a.04o.Z
-rw-r--r-- 1 wcdaop gpsusers 898919 Nov 18 22:10 bcdl320a.04o.zip
Note that the .gz and .zip compressions of the .04o file is slightly better
than the .Z compression but that the opposite is true for the .04d file.
Wrt which one to use, my 2cents(Cdn) worth:
a) it would not appear that there is a huge space saving of one over the
other {from this limited, though highly scientific test ... ahem .. :>) }
b) it would be appropriate to examine which format is best for our user
base, specifically the none-IGS user base, and that would include the use of
the Hatanaka compression (I can't tell you how many phone calls / emails I
have had on that one ...); for the purpose of this discussion I am assuming
that most core IGS data / analysis centres can handle any of the above,
including Hatanaka. However the same cannot necessarily be said for the
non-IGS users:
- commercial programs can uncompress any of the above (except Hatanaka) but
cannot necesssarily re-compress them in that format (based on examining
Winzip's web page).
- what format can commercial GPS packages handle? In other words how can we
make it easier for the non-expert to handle IGS data streams and thus
encourage / increase use of our products?
- is it reasonable to continue with the Hatanaka compression given that
there are few commercial applications that can handle it? - I believe some
of the vendor software packages can but is this universal?
- should keep in mind that many people do not know how to use DOS programs
(including the latest batch of grad students we've had) so in the B.Gates
world, utilities really have to be Windows based; I believe the gzip
utilities are only available for DOS, not Windows (???), similarly Hatanaka
(except as noted above).
c) my suggestion would be the following:
i) status quo for the time being;
ii) do a little bit of research to see what the optimal compression scheme
would be and what tools are available from / supported by the GNSS vendors
and at that point make any adjustments as required.
iii) roll out the change(s) once we are sure that these will address current
/ future needs.
iv) don't bother converting old data files, but leave that up to individual
data centres.
I tend to agree with Angie that .Z may become a "legacy" format and that the
gzip format will probably become more widely used. I would be curious to
hear what the common compression formats outside of N. America are,
especially if different from the above.
end of 2 cents - hope I have not overcharged :>)
cheers
Mike
PS if we had a compression scheme as effective as the proprietaty Trimble
T00 format all of this would be a mute point ....
----------------------------------------------------------------------------
Michael Schmidt, P.Eng. MSchmidt at nrcan.gc.ca
TEL +1 250-363-6760 FAX +1 250-363-6565 Mobile +1 250-920-6128
Geological Survey of Canada Commission géologique du Canada
Natural Resources Canada Ressources naturelles Canada
PO Box 6000 B.P. 6000
Sidney BC Canada V8L 4B2 Sidney (C.B.) Canada V8L 4B2
Street: Rue:
9860 West Saanich Rd 9860 West Saanich Rd
North Saanich BC North Saanich (C.B.)
Canada V8L 3S1 Canada V8L 3S1
Government of Canada Gouvernement du Canada
www.pgc.nrcan.gc.ca
----------------------------------------------------------------------------
> -----Original Message-----
> From: owner-igs-dcwg at igscb.jpl.nasa.gov
> [mailto:owner-igs-dcwg at igscb.jpl.nasa.gov] On Behalf Of Carey Noll
> Sent: Thursday, November 18, 2004 1:10 PM
> To: Angelyn W. Moore
> Cc: igs-dcwg at igscb.jpl.nasa.gov
> Subject: [IGS-DCWG-29] Re: [IGS-DCWG-27] Any DCs accepting .gz RINEX?
>
>
> **************************************************************
> ****************
> IGS-DCWG Mail 18 Nov 13:15:21 PST 2004 Message Number 29
> **************************************************************
> ****************
>
> Author: Carey Noll
>
> Angie,
>
> CDDIS can certainly do a translation of a file from .gz to
> Z as long as we know who is doing it ahead of time so the
> scripts can be modified of course. There may also be some
> delay in data availability by adding this extra step of
> course (minor).
>
> A large archive like CDDIS does not really want to re-do
> the entire archive in another compression scheme! Especially
> since we are trying to get the archive up and running on a
> new server. I don't see the benefit of doing this, but I am
> probably wearing blinders.
>
> Carey.
>
> +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=
> +=+=+=+=+=
> ++
> Ms. Carey Noll Manager, Crustal Dynamics Data Information
> System (CDDIS)
> Secretary, ILRS
> Central Bureau
> Code 922 E-mail:
> Carey.Noll at nasa.gov
> NASA GSFC Voice:
> (301) 614-6542
> Greenbelt, MD 20771 Fax:
> (301) 614-6099
> USA WWW:
> http://cddisa.gsfc.nasa.gov/cddis_welcome.html
>
> +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=
> +=+=+=+=+=
> ++
>
More information about the IGS-DCWG
mailing list