[webservices] New ws-bulkdataselect revision released

[Chad Trabant]

Hi Philip,

We have discussed how to deal with versioned services/results including both of the suggestions you make below.  We landed at using different base URLs for major changes in API or content, either adding "v2" or changing the service name.

There is a balance to be struck between having numerous versions and too little versioning.  I disagree that "every change, even small ones, can and will royally screw your users".  Adding a parameter to a service and nothing else changes does not, in my opinion, warrant a completely new URL.  Likewise, adding a completely new element to XML results without changing the structure of the existing elements does also not warrant a 99% parallel service, wouldn't be very eXtensible if adding details broke a parser.

For the recent update here was the logic.  The change was relatively disruptive, we added a whole new <Network> level to the StationXML.  StationXML is not quite settled but is getting very close, so we don't expect this to happen much more often if at all.  The ws-station service is relatively new, most folks are using it with the Fetch* scripts we provide which were not effected because of the stream processing of XML (i.e. you are one of the few that are parsing it directly) so that limits the disruption.  In short, the impact of changing the results versus changing the URL in this case was not worth having a new service.  If, say, a year from now we need to do that big of a change and the adoption of ws-station is high we would probably make a new version.

StationXML will continue to evolve to match the needs of SCEDC, NCEDC, IRIS and USGS (and maybe the FDSN in the future).  At this point we do not expect large structural changes, but we do expect additions which should not break any real XML parser.

Chad

On Apr 22, 2011, at 6:51 AM, Philip Crotwell wrote:

> Hi Chad
> 
> I guess it is only natural that as these services are new, there will
> be some changes. It looks like this one is backwards compatible, but
> the previous one to the station web service wasn't and would cause
> problems for any other clients out there.
> 
> Have you all given any thought to a longer term versioning for these
> systems? Here is the problem I worry about. I put code into a software
> package to access these web services. My code is obviously tuned to
> the way the web service works at the time I write and test it. Later,
> after I have made a release, IRIS makes a non-backwards compatible
> change. Then, my client breaks and it looks to the rest of the world
> like I have written buggy code and it can't be fixed without a new
> release of my client. I also have little to no forewarning that the
> change is coming, and therefore have little ability to be proactive in
> what I do.
> 
> If you change the IRIS web page, people just notice the change and
> figure things out. They may like or dislike the change, but they can
> adapt. Web services are fundamentally different than web pages. A
> small change that would cause no problem for a person can easily cause
> a web service client to completely fail.
> 
> I guess it feels to me like once you put out a web service that will
> be called by third party clients, you have some obligation to either
> make only backward compatible changes, or to run the old version and
> the new version in parallel for some time to allow an orderly
> transition. Given the development cycle for clients, my guess is this
> might be closer to years than weeks. Although the IDL revision for
> fissures/dhi was never implemented, we addressed this issue in DHI by
> encoding the IDL version number into the name service lookup system,
> so an IDL 1.0 client would be able to find IDL1.0 servers and would
> not accidentally try to connect to IDL 2.0 servers. Obviously web
> services will use different mechanisms, but if they are both going to
> be used and to evolve at the same time, then you have to provide some
> means for the communications protocol contract to remain stable.
> Otherwise every change, even small ones, can and will royally screw
> your users.
> 
> This versioning could easily be accomplished in several ways, either
> putting a version number in the url, say
> http://www.iris.edu/ws/1.0/bulkdataselect/
> or as a parameter
> http://www.iris.edu/ws/bulkdataselect/?version=1.0
> The former has they advantage of you leaving the old server up and
> just adding new ones, while the second allows you to be fancy in your
> implementation to tailor the output to the client. I guess I prefer
> the stability of the first, but as a client writer I can deal with
> either.
> 
> thanks,
> Philip
> 
> On Wed, Apr 20, 2011 at 7:30 PM, Chad Trabant <chad at iris.washington.edu> wrote:
>> 
>> Hello webservice users,
>> 
>> The DMC has updated it's ws-bulkdataselect service:
>> http://www.iris.edu/ws/bulkdataselect/
>> 
>> The new version adds two optional query parameters:
>> 
>> minimumlength
>>    Enforce minimum segment length (seconds).
>>    Only time-series segments of this length or longer will be returned.
>> 
>> longestonly
>>    If specified, only the longest continuous segment for each channel is
>>    returned.
>> 
>> These new parameters can be used to reduce or avoid receiving data
>> containing gaps.
>> The command line FetchBulkData client script that uses this service to
>> request data has been updated with options to specify these new parameters,
>> the latest client scripts are available here:
>> http://www.iris.edu/ws/wsclients/
>> Usage details: like the quality parameter, the two new parameters may be
>> specified in the main selection list or separately as post parameters.
>> For example, this selection will return only the longest data, that is at
>> least one day (86400 seconds) long:
>> 
>> minimumlength 86400.0
>> longestonly
>> TA A25A -- BHZ 2010-084T00:00:00 2010-091T00:00:00
>> TA A25A -- BHE 2010-084T00:00:00 2010-091T00:00:00
>> regards
>> Web services team.
>> 
>> 
>> _______________________________________________
>> webservices mailing list
>> webservices at iris.washington.edu
>> http://www.iris.washington.edu/mailman/listinfo/webservices
>> 
>> 
> 
> _______________________________________________
> webservices mailing list
> webservices at iris.washington.edu
> http://www.iris.washington.edu/mailman/listinfo/webservices