[webservices] includeavailability

[Chad Trabant]

Hi Joachim,

The data availability sub-system at the DMC is updated every 12 hours for archive data.  Data is added to the archive (i.e. from the real time collection system) on a regular basis and is usually all in place ~18 hours after arriving, often sooner, but it varies depending on system load.  The data availability you see our our fdsnws-station service is determined by the combination of those activities.

To report the near real-time data as available when setting matchtimeseries=TRUE, our service assumes that data that has been archived within the last 36 hours is still flowing into the data center.  This is documented in the Data Availability section of the Help for the service: http://service.iris.edu/fdsnws/station/docs/1/help/ <http://service.iris.edu/fdsnws/station/docs/1/help/>

So yes, the data availably information is not in perfect sync with the actual availability, it never will be of course: (in the extreme) by the time you receive and parse the response from our service even the real time information is likely out of date.  So it's really a question of how good is good enough.  Frankly, we have found that the 'matchtimeseries' parameter with the 36 hour rule for near real time data is sufficient for most needs.

If you just want to know what data is available a couple of hours ago, consider using matchtimeseries with a correct time window, and then assume any returned metadata is for channels that have data available.  It'll be pretty darn close to correct.

We would consider putting effort into improving the near real-time data availability information if there is enough need, so I'd be happy to hear from folks that need such information.  Possibly, we could find alternatives, like checking our real-time SeedLink export system, for many use cases.

Chad

> On Nov 26, 2014, at 4:48 AM, Joachim Saul <saul at gfz-potsdam.de> wrote:
> 
> Hello,
> 
> how frequently is the availability information updated at the IRIS server side? If I do now an inventory request using e.g.
> http://service.iris.edu/fdsnws/station/1/query?network=IU&station=ANMO&channel=BHZ&location=10&level=channel&includeavailability=TRUE&start=2014-11-26T12:00:00
> then I get as availability time span for that stream
> 
> <DataAvailability><Extent start="1998-10-26T20:35:58" end="2014-11-25T12:00:00"/></DataAvailability>
> 
> which ends about a day ago. However, the data are there (of course) and I can retrieve them using
> http://service.iris.edu/fdsnws/dataselect/1/query?network=IU&station=ANMO&channel=BHZ&location=10&start=2014-11-26T12:00:00&end=2014-11-26T12:10:00
> 
> In other words the availability information is not in sync with the actual data holdings. Not just for IU.ANMO but for many more stations/networks. Is this intentional? It currently prevents me from making use of the otherwise very useful availability info when requesting data from a few hours ago.
> 
> Regards
> Joachim
> _______________________________________________
> webservices mailing list
> webservices at iris.washington.edu
> http://www.iris.washington.edu/mailman/listinfo/webservices

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.iris.washington.edu/pipermail/webservices/attachments/20141126/e2f09b9d/attachment.html>