[IGS-DCWG-190] Re: [IGS-IC-816] Re: compression file changes ... [SEC=UNCLASSIFIED]

Wed Jan 9 23:33:23 PST 2019

Hi Nacho and colleagues,

from an AC (or user of the DC facilities) perspective, the change from 
.Z to .gz should not be an issue. We at CODE have changed the 
compression program already for our internal archive.

Regarding your point 2) I understand the concerns on the DC side to 
change the files in the archive (what was already a reasonable effort 
only with our internal archive at CODE). On the other hand, it is from 
the user perspective from very comfortable if one has to change the 
search pattern for downloading a file at a certain date and also the 
internal pre-processing (meaning decompression).

A related question is regarding the products form MGEX with long 
filenames. Should they also be provided (and submitted?) with gzip (to 
be consistent there)?

Best regards

Rolf

Am 10.01.2019 um 04:00 schrieb Ruddick Ryan:
> Hi Nacho.
> 
> A bit of background first.
> 
> At Geoscience Australia we have recently completed a new GNSS data 
> repository (I will send an email announcing this shortly). Early on in 
> the development we hit the problem of UNIX compression not being 
> supported by the modern systems/applications, we spent some time writing 
> the code in Python to support the recompression but in the decided 
> against using it. As such we made a decision to transition all of our 
> historic RINEX 2 data to gzip. We are still submitting files to IGS data 
> centres using UNIX compression and making a copy of files available via 
> old FTP (not via our new data repository, which is not going to support 
> FTP). We do at some point in the near future plan on only supporting and 
> submitting RINEX 3 files.
> 
> In answer to your questions.
> 
> 1.Geoscience Australia is supportive of the change and have no problem 
> with the proposed date. We are in a position to submit RINEX files to 
> IGS DC’s in any compression format requested (our preference is for gzip).
> 
> 2.On a global data centre level I agree that changing the compression of 
> historic files could cause problems. That said at a local/regional 
> level, as stated above we will (have) change(d) all of our historical 
> files to use gzip compression. To continue supporting legacy 
> applications/users we will retain a UNIX compressed copy of the files 
> available only via FTP on our old data centre.
> 
> 3.I am not aware of any specific place, but there may be commercial 
> software packages that will need changes made to them. This could cause 
> problems with legacy software where patching support is no longer provided.
> 
> Regards,
> 
> Ryan
> 
> UNCLASSIFIED
> 
> Classified by ryan.ruddick at ga.gov.au on 10/01/2019 2:00:21 PM
> 
> *From:*IGS-IC <igs-ic-bounces at lists.igs.org> *On Behalf Of *Nacho Romero
> *Sent:* Tuesday, 8 January 2019 9:28 PM
> *To:* igs-dcwg at lists.igs.org; ic IGS List <igs-ic at lists.igs.org>; Lou 
> Estey <lou at unavco.org>; David Maggert <dmaggert at unavco.org>; MacLeod, 
> Ken <Ken.MacLeod at NRCan-RNCan.gc.ca>
> *Cc:* gsfc-cddis-ops at lists.nasa.gov; Limbacher, Rebecca Irene. 
> (GSFC-690.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] 
> <rebecca.i.limbacher at nasa.gov>; mark.van.kints-esa.int 
> <mark.van.kints at esa.int>
> *Subject:* [IGS-IC-814] compression file changes ...
> 
> Dear all,
> 
> At the last IGS workshop at the IC+DC splinter meeting we approved the 
> recommendation ...
> 
> *2018-4. *To coordinate DC, IC and NC so as to move away from Z 
> compression for all IGS files (data and products) and into using gzip (.gz)
> 
> Please provide us comments on the following;
> 
> 1) I propose that we announce this change now with the date 01 June 2019 
> , and at that point we publish all the newly submitted files (data and 
> products) as .gz in all IGS DCs, whether the owner submits it that way 
> or not ... do you agree that DCs should enforce such a change or should 
> we insist all the owners transition their own submitted files?
> 
> 2) Do we change all the historical files to .gz ? This will generate 
> "new files" which the myriad of sync processes (private and public) 
> running between all sort of servers will cause a lot of duplications ... 
> I propose we leave the old files as they are for now to avoid a lot of 
> potential duplication of files in IGS and user's servers
> 
> 3) are you aware of specific places were the .Z compression is 
> requested? required? recommended? ... I think the site guidelines are 
> generic enough, what do you think?
> 
> We have some other recommendations to discuss with you in the next few 
> days, and in the next months I will organize a telecon so we can all 
> agree a way forward for each one.
> 
> Please let us know your comments on this specific recommendation and I 
> will collate all the discussion points for the coordination telecon. Thanks.
> 
> -- 
> 
> Best regards,
> 
> Nacho
> 
> _______________________________________________________________
> 
> Ignacio (Nacho) Romero
> 
> Aerospace Engineer, PhD
> 
> CSC Ltd @ ESA/ESOC/OPS-GN
> 
> Satellite Navigation Engineer @ ESOC Navigation Support Office
> 
> IGS Infrastructure Committee Chair
> 
> www.navigation-office.esa.int  <http://www.navigation-office.esa.int>
> 
> www.canaryspaceconsulting.co.uk  <http://www.canaryspaceconsulting.co.uk>
> 
> _______________________________________________________________
> 
> This message including any attachments may contain confidential
> 
> information, according to our Information Security Management System,
> 
>   and intended solely for a specific individual to whom they are addressed.
> 
>   Any unauthorised copy, disclosure or distribution of this message
> 
>   is strictly forbidden. If you have received this transmission in error,
> 
>   please notify the sender immediately and delete it.
> 
> Geoscience Australia Disclaimer: This e-mail (and files transmitted with 
> it) is intended only for the person or entity to which it is addressed. 
> If you are not the intended recipient, then you have received this 
> e-mail by mistake and any use, dissemination, forwarding, printing or 
> copying of this e-mail and its file attachments is prohibited. The 
> security of emails transmitted cannot be guaranteed; by forwarding or 
> replying to this email, you acknowledge and accept these risks.
> -------------------------------------------------------------------------------------------------------------------------
> 
> 
> _______________________________________________
> IGS-IC mailing list
> IGS-IC at lists.igs.org
> https://lists.igs.org/mailman/listinfo/igs-ic
> 

-- 
----------------------------------------------------------------------
Rolf Dach                   Astronomical Institute, University of Bern
rolf.dach at aiub.unibe.ch     Sidlerstrasse 5, CH-3012 Bern, Switzerland
http://www.aiub.unibe.ch/   Tel: +41-31-631 8593
ftp://ftp.aiub.unibe.ch
----------------------------------------------------------------------