[IGS-DCWG-194] Re: [IGS-IC-814] compression file changes ...

[Elisabetta D'Anastasio]

Dear Nacho (et al),

Some thoughts from the GeoNet (NZ) data centre point of view.


  1.  Date: 01 (or 02, based on the GPS week) June 2019 sounds good, and we are very close to switch to a new internal system to handle GNSS data that produces only gzip compressed rinex (a communication about that will come shortly).
  2.  Archive: Although is a giant effort on several different levels, I think that changing compression format on the whole archive will guarantee future data access. The GeoNet archive currently has made this already, and we presently have both d.Z and o.gz formats in our data storage (more on that below).
  3.  nothing that we are aware of

I’d be interested to participate to the telecon, and, I’m interested also in Brian’s comment around compression of data products in general.

Also, relatively to the compression discussion, I would like to ask what the plans are in terms of Hatanaka compression.
With modern internet networks, the requirement of compressing the rinex plain text is probably less important.
Although file size is still significantly reduced by applying hatanaka compression, we have decided to move away from that in order to guarantee future access to the rinex data and decouple that from the RNX2CRX software package.

Hope that helps the conversation,

Kind regards,

Elisabetta



Elisabetta D’Anastasio I Geodetic Processing Specialist
GeoNet, GeoHazard Monitoring Department
GNS Science I Te Pῡ Ao
1 Fairway Drive, Avalon 5010, PO Box 30368, Lower Hutt 5040, New Zealand
Ph +64 (0) 4 570 4744 I Mob +64 (02) 27 6655 890
http://www.geonet.org.nz/<http://www.gns.cri.nz/> I Email: e.danastasio at gns.cri.nz<mailto:e.danastasio at gns.cri.nz>





From: IGS-IC <igs-ic-bounces at lists.igs.org<mailto:igs-ic-bounces at lists.igs.org>> On Behalf Of Nacho Romero
Sent: January 8, 2019 5:28 AM
To: igs-dcwg at lists.igs.org<mailto:igs-dcwg at lists.igs.org>; ic IGS List <igs-ic at lists.igs.org<mailto:igs-ic at lists.igs.org>>; Lou Estey <lou at unavco.org<mailto:lou at unavco.org>>; David Maggert <dmaggert at unavco.org<mailto:dmaggert at unavco.org>>; MacLeod, Ken (NRCan/RNCan) <ken.macleod at canada.ca<mailto:ken.macleod at canada.ca>>
Cc: gsfc-cddis-ops at lists.nasa.gov<mailto:gsfc-cddis-ops at lists.nasa.gov>; Limbacher, Rebecca Irene. (GSFC-690.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] <rebecca.i.limbacher at nasa.gov<mailto:rebecca.i.limbacher at nasa.gov>>; mark.van.kints-esa.int <mark.van.kints at esa.int<mailto:mark.van.kints at esa.int>>
Subject: [IGS-IC-814] compression file changes ...


Dear all,

At the last IGS workshop at the IC+DC splinter meeting we approved the recommendation ...

2018-4. To coordinate DC, IC and NC so as to move away from Z compression for all IGS files (data and products) and into using gzip (.gz)



Please provide us comments on the following;

1) I propose that we announce this change now with the date 01 June 2019 , and at that point we publish all the newly submitted files (data and products) as .gz in all IGS DCs, whether the owner submits it that way or not ... do you agree that DCs should enforce such a change or should we insist all the owners transition their own submitted files?



2) Do we change all the historical files to .gz ? This will generate "new files" which the myriad of sync processes (private and public) running between all sort of servers will cause a lot of duplications ... I propose we leave the old files as they are for now to avoid a lot of potential duplication of files in IGS and user's servers

3) are you aware of specific places were the .Z compression is requested? required? recommended? ... I think the site guidelines are generic enough, what do you think?



We have some other recommendations to discuss with you in the next few days, and in the next months I will organize a telecon so we can all agree a way forward for each one.

Please let us know your comments on this specific recommendation and I will collate all the discussion points for the coordination telecon. Thanks.

--

Best regards,

Nacho

_______________________________________________________________



Ignacio (Nacho) Romero

Aerospace Engineer, PhD

CSC Ltd @ ESA/ESOC/OPS-GN

Satellite Navigation Engineer @ ESOC Navigation Support Office

IGS Infrastructure Committee Chair

www.navigation-office.esa.int<http://www.navigation-office.esa.int>

www.canaryspaceconsulting.co.uk<http://www.canaryspaceconsulting.co.uk>

_______________________________________________________________

This message including any attachments may contain confidential

information, according to our Information Security Management System,

 and intended solely for a specific individual to whom they are addressed.

 Any unauthorised copy, disclosure or distribution of this message

 is strictly forbidden. If you have received this transmission in error,

 please notify the sender immediately and delete it.

Notice: This email and any attachments are confidential and may not be used, published or redistributed without the prior written consent of the Institute of Geological and Nuclear Sciences Limited (GNS Science). If received in error please destroy and immediately notify GNS Science. Do not copy or disclose the contents.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.igs.org/pipermail/igs-dcwg/attachments/20190111/af74aff2/attachment-0001.html>