John West
Hi.
The query
http://www.iris.edu/ws/station/query?updatedafter=2011-06-06&level=sta&net=ND&timewindow=2010-01-01,2500-12-12
returns multiple duplicates of the <StationEpoch> sections for each
station.
Makes my sync to a local copy procedure very slow...
Thanks!
-- John
The query
http://www.iris.edu/ws/station/query?updatedafter=2011-06-06&level=sta&net=ND&timewindow=2010-01-01,2500-12-12
returns multiple duplicates of the <StationEpoch> sections for each
station.
Makes my sync to a local copy procedure very slow...
Thanks!
-- John
-
Hi John,
What happened is that you selected "station level" information using the -sta flag. If you didn't use that, it would default to channel level and you would see that WS is returning one line for each channel and omitting the channel name, so it looks like lots of duplicates. If you want only the earliest and latest station epochs, you will have to read them all to figure this out.
Also just as an FYI since you are interested in the epochs, sometimes a very tiny gap in station epochs can exist (like one sample point e.g. 0.05sec) or metadata gets changed and what happens is that you will see 2 epochs where the end of the first epoch will equal the start of the second.
epoch 1: 2002-03-17T00:00:00,2005-04-05T06:07:08
epoch 2: 2005-04-05T06:07:08,2599-12-31T23:59:59
Alex
channel level:
scoop synthetics/METADATA> FetchMetadata -ua 2011-06-06 -N ND -vvv -s 2010-01-01 -e 2500-12-12
FetchMetadata (2011.172) at 2011-11-13 18:37:39
== Data selections ==
ND||||2010-01-01T00:00:00|2500-12-12T00:00:00
Metadata URI: 'http://www.iris.edu/ws/station/query?level=chan&network=ND&timewindow=2010-01-01,2500-12-12&updatedafter=2011-06-06'
Fetching metadata :: Received 108.1 KB
Received 108.1 KB of metadata in 0.1 seconds (1.2 MB/s)
Parsing XML metadata... Done (0.1 seconds)
Processed metadata for 84 channel epochs in 0.2 seconds (488.3 KB/s)
Writing metadata (84 channel epochs) file
#net,sta,loc,chan,lat,lon,elev,depth,azimuth,SACdip,instrument,scale,scalefreq,scaleunits,samplerate,start,end
ND,KOUNC,--,BHE,-20.5507,164.2861,88.0,0.0,90.0,90,Trillium 120P; 120 s; 1201 V/m/s-Kephren; gain 1;,1.28257E11,0.05,M/S,20,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,BHN,-20.5507,164.2861,88.0,0.0,0.0,90,Trillium 120P; 120 s; 1201 V/m/s-Kephren; gain 1;,1.28257E11,0.05,M/S,20,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,BHZ,-20.5507,164.2861,88.0,0.0,0.0,0,Trillium 120P; 120 s; 1201 V/m/s-Kephren; gain 1;,1.28257E11,0.05,M/S,20,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,BNE,-20.5507,164.2861,88.0,0.0,90.0,90,Episensor; DC to >200 Hz; 20V differential full s,2.18979E8,0.05,M/S**2,20,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,BNN,-20.5507,164.2861,88.0,0.0,0.0,90,Episensor; DC to >200 Hz; 20V differential full s,2.18979E8,0.05,M/S**2,20,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,BNZ,-20.5507,164.2861,88.0,0.0,0.0,0,Episensor; DC to >200 Hz; 20V differential full s,2.18979E8,0.05,M/S**2,20,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,HHE,-20.5507,164.2861,88.0,0.0,90.0,90,Trillium 120P; 120 s; 1201 V/m/s-Kephren; gain 1;,1.28257E11,0.05,M/S,100,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,HHN,-20.5507,164.2861,88.0,0.0,0.0,90,Trillium 120P; 120 s; 1201 V/m/s-Kephren; gain 1;,1.28257E11,0.05,M/S,100,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,HHZ,-20.5507,164.2861,88.0,0.0,0.0,0,Trillium 120P; 120 s; 1201 V/m/s-Kephren; gain 1;,1.28257E11,0.05,M/S,100,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,HNE,-20.5507,164.2861,88.0,0.0,90.0,90,Episensor; DC to >200 Hz; 20V differential full s,2.18979E8,0.05,M/S**2,100,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,HNN,-20.5507,164.2861,88.0,0.0,0.0,90,Episensor; DC to >200 Hz; 20V differential full s,2.18979E8,0.05,M/S**2,100,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,HNZ,-20.5507,164.2861,88.0,0.0,0.0,0,Episensor; DC to >200 Hz; 20V differential full s,2.18979E8,0.05,M/S**2,100,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,LIFNC,--,BHE,-20.7771,167.2402,33.0,0.0,90.0,90,Trillium 120P; 120 s; 1201 V/m/s-Kephren; gain 1;,1.28257E11,0.05,M/S,20,2010-08-28T00:00:00,2599-12-31T23:59:59
ND,LIFNC,--,BHN,-20.7771,167.2402,33.0,0.0,0.0,90,Trillium 120P; 120 s; 1201 V/m/s-Kephren; gain 1;,1.28257E11,0.05,M/S,20,2010-08-28T00:00:00,2599-12-31T23:59:59
ND,LIFNC,--,BHZ,-20.7771,167.2402,33.0,0.0,0.0,0,Trillium 120P; 120 s; 1201 V/m/s-Kephren; gain 1;,1.28257E11,0.05,M/S,20,2010-08-28T00:00:00,2599-12-31T23:59:59
ND,LIFNC,--,BNE,-20.7771,167.2402,33.0,0.0,90.0,90,Episensor; DC to >200 Hz; 20V differential full s,2.18979E8,0.05,M/S**2,20,2010-08-28T00:00:00,2599-12-31T23:59:59
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
station level (-sta flag):
scoop synthetics/METADATA> FetchMetadata -ua 2011-06-06 -N ND -vvv -s 2010-01-01 -e 2500-12-12 -sta
FetchMetadata (2011.172) at 2011-11-13 18:36:27
== Data selections ==
ND||||2010-01-01T00:00:00|2500-12-12T00:00:00
Metadata URI: 'http://www.iris.edu/ws/station/query?level=sta&network=ND&timewindow=2010-01-01,2500-12-12&updatedafter=2011-06-06'
Fetching metadata :: Received 38.3 KB
Received 38.3 KB of metadata in 0.1 seconds (492.5 KB/s)
Parsing XML metadata... Done (0.0 seconds)
Processed metadata for 84 station epochs in 0.1 seconds (323.3 KB/s)
Writing metadata (84 station epochs) file
#net,sta,lat,lon,elev,site,start,end
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,LIFNC,-20.7771,167.2402,33.0,Lifou; Loyalty Islands; New Caledonia,2010-08-28T00:00:00,2599-12-31T23:59:59
ND,LIFNC,-20.7771,167.2402,33.0,Lifou; Loyalty Islands; New Caledonia,2010-08-28T00:00:00,2599-12-31T23:59:59
ND,LIFNC,-20.7771,167.2402,33.0,Lifou; Loyalty Islands; New Caledonia,2010-08-28T00:00:00,2599-12-31T23:59:59
ND,LIFNC,-20.7771,167.2402,33.0,Lifou; Loyalty Islands; New Caledonia,2010-08-28T00:00:00,2599-12-31T23:59:59
On Nov 13, 2011, at 4:05 PM, John D. West wrote:
Hi.
The query http://www.iris.edu/ws/station/query?updatedafter=2011-06-06&level=sta&net=ND&timewindow=2010-01-01,2500-12-12
returns multiple duplicates of the <StationEpoch> sections for each station.
Makes my sync to a local copy procedure very slow...
Thanks!
-- John
_______________________________________________
webservices mailing list
webservices<at>iris.washington.edu
http://www.iris.washington.edu/mailman/listinfo/webservices
-
Hi, Alex.
OK, I understand why I'm getting duplicates. I do think, however, that the
service should only return distinct StationEpochs. If the service returns
this level of duplicated information, it makes me wonder whether the
level=sta format has any utility. Why even provide it?
Thanks!
-- John
On Mon, Nov 14, 2011 at 1:16 PM, Alex <alex<at>iris.washington.edu> wrote:
Hi John,
What happened is that you selected "station level" information using the
-sta flag. If you didn't use that, it would default to channel level and
you would see that WS is returning one line for each channel and omitting
the channel name, so it looks like lots of duplicates. If you want only
the earliest and latest station epochs, you will have to read them all to
figure this out.
Also just as an FYI since you are interested in the epochs, sometimes a
very tiny gap in station epochs can exist (like one sample point e.g.
0.05sec) or metadata gets changed and what happens is that you will see 2
epochs where the end of the first epoch will equal the start of the second.
epoch 1: 2002-03-17T00:00:00,2005-04-05T06:07:08
epoch 2: 2005-04-05T06:07:08,2599-12-31T23:59:59
Alex
channel level:
scoop synthetics/METADATA> FetchMetadata -ua 2011-06-06 -N ND -vvv -s
2010-01-01 -e 2500-12-12
FetchMetadata (2011.172) at 2011-11-13 18:37:39
== Data selections ==
ND||||2010-01-01T00:00:00|2500-12-12T00:00:00
Metadata URI: '
http://www.iris.edu/ws/station/query?level=chan&network=ND&timewindow=2010-01-01,2500-12-12&updatedafter=2011-06-06
'
Fetching metadata :: Received 108.1 KB
Received 108.1 KB of metadata in 0.1 seconds (1.2 MB/s)
Parsing XML metadata... Done (0.1 seconds)
Processed metadata for 84 channel epochs in 0.2 seconds (488.3 KB/s)
Writing metadata (84 channel epochs) file
#net,sta,loc,chan,lat,lon,elev,depth,azimuth,SACdip,instrument,scale,scalefreq,scaleunits,samplerate,start,end
ND,KOUNC,--,BHE,-20.5507,164.2861,88.0,0.0,90.0,90,Trillium 120P; 120 s;
1201 V/m/s-Kephren; gain
1;,1.28257E11,0.05,M/S,20,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,BHN,-20.5507,164.2861,88.0,0.0,0.0,90,Trillium 120P; 120 s;
1201 V/m/s-Kephren; gain
1;,1.28257E11,0.05,M/S,20,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,BHZ,-20.5507,164.2861,88.0,0.0,0.0,0,Trillium 120P; 120 s;
1201 V/m/s-Kephren; gain
1;,1.28257E11,0.05,M/S,20,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,BNE,-20.5507,164.2861,88.0,0.0,90.0,90,Episensor; DC to >200
Hz; 20V differential full
s,2.18979E8,0.05,M/S**2,20,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,BNN,-20.5507,164.2861,88.0,0.0,0.0,90,Episensor; DC to >200
Hz; 20V differential full
s,2.18979E8,0.05,M/S**2,20,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,BNZ,-20.5507,164.2861,88.0,0.0,0.0,0,Episensor; DC to >200 Hz;
20V differential full
s,2.18979E8,0.05,M/S**2,20,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,HHE,-20.5507,164.2861,88.0,0.0,90.0,90,Trillium 120P; 120 s;
1201 V/m/s-Kephren; gain
1;,1.28257E11,0.05,M/S,100,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,HHN,-20.5507,164.2861,88.0,0.0,0.0,90,Trillium 120P; 120 s;
1201 V/m/s-Kephren; gain
1;,1.28257E11,0.05,M/S,100,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,HHZ,-20.5507,164.2861,88.0,0.0,0.0,0,Trillium 120P; 120 s;
1201 V/m/s-Kephren; gain
1;,1.28257E11,0.05,M/S,100,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,HNE,-20.5507,164.2861,88.0,0.0,90.0,90,Episensor; DC to >200
Hz; 20V differential full
s,2.18979E8,0.05,M/S**2,100,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,HNN,-20.5507,164.2861,88.0,0.0,0.0,90,Episensor; DC to >200
Hz; 20V differential full
s,2.18979E8,0.05,M/S**2,100,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,--,HNZ,-20.5507,164.2861,88.0,0.0,0.0,0,Episensor; DC to >200 Hz;
20V differential full
s,2.18979E8,0.05,M/S**2,100,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,LIFNC,--,BHE,-20.7771,167.2402,33.0,0.0,90.0,90,Trillium 120P; 120 s;
1201 V/m/s-Kephren; gain
1;,1.28257E11,0.05,M/S,20,2010-08-28T00:00:00,2599-12-31T23:59:59
ND,LIFNC,--,BHN,-20.7771,167.2402,33.0,0.0,0.0,90,Trillium 120P; 120 s;
1201 V/m/s-Kephren; gain
1;,1.28257E11,0.05,M/S,20,2010-08-28T00:00:00,2599-12-31T23:59:59
ND,LIFNC,--,BHZ,-20.7771,167.2402,33.0,0.0,0.0,0,Trillium 120P; 120 s;
1201 V/m/s-Kephren; gain
1;,1.28257E11,0.05,M/S,20,2010-08-28T00:00:00,2599-12-31T23:59:59
ND,LIFNC,--,BNE,-20.7771,167.2402,33.0,0.0,90.0,90,Episensor; DC to >200
Hz; 20V differential full
s,2.18979E8,0.05,M/S**2,20,2010-08-28T00:00:00,2599-12-31T23:59:59
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
station level (-sta flag):
scoop synthetics/METADATA> FetchMetadata -ua 2011-06-06 -N ND -vvv -s
2010-01-01 -e 2500-12-12 -sta
FetchMetadata (2011.172) at 2011-11-13 18:36:27
== Data selections ==
ND||||2010-01-01T00:00:00|2500-12-12T00:00:00
Metadata URI: '
http://www.iris.edu/ws/station/query?level=sta&network=ND&timewindow=2010-01-01,2500-12-12&updatedafter=2011-06-06
'
Fetching metadata :: Received 38.3 KB
Received 38.3 KB of metadata in 0.1 seconds (492.5 KB/s)
Parsing XML metadata... Done (0.0 seconds)
Processed metadata for 84 station epochs in 0.1 seconds (323.3 KB/s)
Writing metadata (84 station epochs) file
#net,sta,lat,lon,elev,site,start,end
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New
Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New
Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New
Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New
Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New
Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New
Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New
Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New
Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New
Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New
Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New
Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,KOUNC,-20.5507,164.2861,88.0,Koumac; New
Caledonia,2011-03-17T00:00:00,2599-12-31T23:59:59
ND,LIFNC,-20.7771,167.2402,33.0,Lifou; Loyalty Islands; New
Caledonia,2010-08-28T00:00:00,2599-12-31T23:59:59
ND,LIFNC,-20.7771,167.2402,33.0,Lifou; Loyalty Islands; New
Caledonia,2010-08-28T00:00:00,2599-12-31T23:59:59
ND,LIFNC,-20.7771,167.2402,33.0,Lifou; Loyalty Islands; New
Caledonia,2010-08-28T00:00:00,2599-12-31T23:59:59
ND,LIFNC,-20.7771,167.2402,33.0,Lifou; Loyalty Islands; New
Caledonia,2010-08-28T00:00:00,2599-12-31T23:59:59
On Nov 13, 2011, at 4:05 PM, John D. West wrote:
Hi.
The query
http://www.iris.edu/ws/station/query?updatedafter=2011-06-06&level=sta&net=ND&timewindow=2010-01-01,2500-12-12
returns multiple duplicates of the <StationEpoch> sections for each
station.
Makes my sync to a local copy procedure very slow...
Thanks!
-- John
_______________________________________________
webservices mailing list
webservices<at>iris.washington.edu
http://www.iris.washington.edu/mailman/listinfo/webservices
-
Hi
This might be related to the duplicate <stationEpoch> issue I found on
Oct 19. Yazan said that this was a bug and would be fixed in the next
version.
Philip
------------
Thanks Philip
This is a bug and will be addressed in the new release (1.2.6). The
new release is scheduled for release very soon. Yazan
On Oct 19, 2011, at 7:08 AM, Philip Crotwell wrote:
Hi
This URL generates a large number of duplicate <stationEpoch> elements
for each station. Oddly, if anything, including a *, is put in the
"sta" query item, then the duplicate stationEpoch elements are not
there. Deleting the timewindow also removes the duplicate entries. Not
sure, but wonder if it is picking up a stationEpoch for each channel?
http://www.iris.edu/ws/station/query?net=II&timewindow=1997-06-07%2C2011-06-07&level=sta
thanks,
Philip
-
-
-
Hi John,
We have beta release ready to go out that fixes this bug and others. Unless remaining issues are found the beta will be released for external testing today.
Chad
On Nov 13, 2011, at 4:05 PM, John D. West wrote:
Hi.
The query http://www.iris.edu/ws/station/query?updatedafter=2011-06-06&level=sta&net=ND&timewindow=2010-01-01,2500-12-12
returns multiple duplicates of the <StationEpoch> sections for each station.
Makes my sync to a local copy procedure very slow...
Thanks!
-- John
_______________________________________________
webservices mailing list
webservices<at>iris.washington.edu
http://www.iris.washington.edu/mailman/listinfo/webservices
Alex Hutko
Philip Crotwell
Chad Trabant
