-
Notifications
You must be signed in to change notification settings - Fork 15
/
README.Rmd
388 lines (314 loc) · 19.6 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
---
output: github_document
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README-",
out.width = "100%"
)
```
# {GSODR}: Global Surface Summary of the Day (GSOD) Weather Data Client <img src="man/figures/logo.png" style="float:right;" alt="logo" width="120" />
<!-- badges: start -->
[](https://github.com/ropensci/GSODR/actions/workflows/R-CMD-check.yaml)
[](https://app.codecov.io/gh/ropensci/GSODR)
[](https://doi.org/10.5281/zenodo.439850)
[](https://cran.r-project.org/package=GSODR)
[](https://www.repostatus.org/#active)
[](https://joss.theoj.org/papers/10.21105/joss.00177)
[](https://github.com/ropensci/software-review/issues/79)
<!-- badges: end -->
## Introduction
The GSOD or [Global Surface Summary of the Day (GSOD)](https://www.ncei.noaa.gov/access/metadata/landing-page/bin/iso?id=gov.noaa.ncdc:C00516) data provided by the US National Centers for Environmental Information (NCEI) are a valuable source of weather data with global coverage.
{GSODR} aims to make it easy to find, transfer and format the data you need for use in analysis and provides six main functions for facilitating this:
- `get_GSOD()` - this function queries and transfers files from the NCEI's web server, reformats them and returns a data frame.
- `reformat_GSOD()` - this function takes individual station files from the local disk and re-formats them returning a data frame.
- `nearest_stations()` - this function returns a `data.table` of stations with their metadata and the distance in which they fall from the given radius (kilometres) of a point given as latitude and longitude in order from nearest to farthest.
- `get_inventory()` - this function downloads the latest station inventory information from the NCEI's server and returns the header information about the latest version as a message in the console and a tidy data frame of the stations' inventory for each month that data are reported.
- `get_history()` - this function downloads the latest version of the isd-history.csv file from the NCEI's server and returns a {data.table} of the information for each station that is available. A version of this file is distributed with {GSODR} internally and can be updated with `update_internal_isd_history()`.
- `get_updates()` - this function downloads the changelog for the GSOD data from the NCEI's server and reorders it by the most recent changes first.
- `update_internal_isd_history()` - this function downloads the latest station list from the NCEI's server updates the package's internal database of stations and their metadata. **Not recommended for normal use.**
When reformatting data either with `get_GSOD()` or `reformat_GSOD()`, all units are converted to International System of Units (SI), _e.g._, inches to millimetres and Fahrenheit to Celsius.
File output is returned as a `data.table` object, summarising each year by station, which also includes vapour pressure and relative humidity elements calculated from existing data in GSOD.
Additional data are calculated by this R package using the original data and included in the final data.
These include vapour pressure (ea and es) and relative humidity calculated using the improved August-Roche-Magnus approximation (Alduchov and Eskridge 1996).
For more information see the description of the data provided by NCEI, <https://www.ncei.noaa.gov/data/global-summary-of-the-day/doc/readme.txt>.
## How to Install
### Stable Version
A stable version of {GSODR} is available from [CRAN](https://cran.r-project.org/package=GSODR).
```r
install.packages("GSODR")
```
### Development Version
A development version is available from from GitHub.
If you wish to install the development version that may have new features or bug fixes before the CRAN version does (but also may not work properly), please install from the [rOpenSci R Universe](https://ropensci.org/r-universe/).
We strive to keep the main branch on GitHub functional and working properly.
```r
install.packages("GSODR", repos = "https://ropensci.r-universe.dev")
```
## Using {GSODR}
The most common work might be getting data for a single location.
Here's an example of fetching data for a station in Toowoomba, Queensland, AU in 2021.
```{r example}
library(GSODR)
tbar <- get_GSOD(years = 2021, station = "955510-99999")
tbar
```
## Other Sources of Weather Data in R
There are several other sources of weather data and ways of retrieving them through R.
Several are also [rOpenSci](https://ropensci.org) projects.
[{clifro}](https://CRAN.R-project.org/package=clifro) from [rOpenSci](https://docs.ropensci.org/clifro/) is a web portal to the New Zealand National Climate Database and provides public access (via subscription) to around 6,500 various climate stations (see <https://cliflo.niwa.co.nz/> for more information).
Collating and manipulating data from CliFlo (hence clifro) and importing into R for further analysis, exploration and visualisation is now straightforward and coherent.
The user is required to have an Internet connection, and a current CliFlo subscription (free) if data from stations, other than the public Reefton electronic weather station, is sought.
[{GSODTools}](https://github.com/environmentalinformatics-marburg/GSODTools) by [Florian Detsch](https://github.com/fdetsch) is an R package that offers similar functionality as {GSODR}, but also has the ability to graph the data and working with data for time series analysis.
[{nasapower}](https://CRAN.R-project.org/package=nasapower) from [rOpenSci](https://docs.ropensci.org/nasapower/) aims to make it quick and easy to automate downloading of the NASA-POWER global meteorology, surface solar energy and climatology data in your R session as a tidy `tibble` object for analysis and use in modelling or other purposes.
POWER (Prediction Of Worldwide Energy Resource) data are freely available for download with varying spatial resolutions dependent on the original data and with several temporal resolutions depending on the POWER parameter and community.
[{riem}](https://CRAN.R-project.org/package=riem) from [rOpenSci](https://docs.ropensci.org/riem/) allows to get weather data from Automated Surface Observing System (ASOS) stations (airports) in the whole world thanks to the Iowa Environment Mesonet website.
[{rnoaa}](https://CRAN.R-project.org/package=rnoaa), from [rOpenSci](https://docs.ropensci.org/rnoaa/) offers tools for interacting with and downloading weather data from the United States National Oceanic and Atmospheric Administration but lacks support for GSOD data.
[{stationaRy}](https://cran.r-project.org/package=stationaRy), from Richard Iannone offers hourly meteorological data from stations located all over the world.
There is a wealth of data available, with historic weather data accessible from nearly 30,000 stations.
[{weathercan}](https://CRAN.R-project.org/package=weathercan) from [rOpenSci](https://github.com/ropensci/weathercan) makes it easier to search for and download multiple months/years of historical weather data from Environment and Climate Change Canada (ECCC) website.
[{weatherOz}](https://CRAN.R-project.org/package=weatherOz) aims to facilitate access and download weather and climate data for Australia from Australian data sources.
Data are sourced from from the Western Australian Department of Primary Industries and Regional Development (DPIRD) and the Scientific Information for Land Owners (SILO) API endpoints and the Australian Government Bureau of Meteorology’s (BOM) FTP server.
[{worldmet}](https://CRAN.R-project.org/package=worldmet) provides an easy way to access data from the NOAA Integrated Surface Database (ISD) (the same database {GSODR} provides access to.
The ISD contains detailed surface meteorological data from around the world for over 35,000 locations.
However, rather than daily values, the package outputs (typically hourly meteorological data) and works very well with the [{openair}](https://CRAN.R-project.org/package=openair) package.
## Notes
### Citing GSOD data
> Cite as: NOAA National Centers of Environmental Information. 1999. Global Surface Summary of the Day - GSOD. 1.0. [indicate subset used]. NOAA National Centers for Environmental Information. Accessed [date].
### NOAA policy
Users of these data should take into account the following:
> The data summaries provided here are based on data exchanged under the World Meteorological Organization (WMO) World Weather Watch Program according to WMO Resolution 40 (Cg-XII). This allows WMO member countries to place restrictions on the use or re-export of their data for commercial purposes outside of the receiving country.
Data for selected countries may, at times, not be available through this system.
Those countries' data summaries and products which are available here are intended for free and unrestricted use in research, education, and other non-commercial activities.
However, for non-U.S. locations' data, the data or any derived product shall not be provided to other users or be used for the re-export of commercial services.
## Meta
- Please [report any issues or bugs](https://github.com/ropensci/GSODR/issues).
- License: MIT
- To cite {GSODR}, please use: Adam H. Sparks, Tomislav Hengl and Andrew Nelson (2017). GSODR: Global Summary Daily Weather Data in R. _The Journal of Open Source Software_, **2(10)**. DOI: 10.21105/joss.00177.
## Code of Conduct
Please note that this package is released with a [Contributor Code of Conduct](https://ropensci.org/code-of-conduct/).
By contributing to this project, you agree to abide by its terms.
## References
Alduchov, O.A. and Eskridge, R.E., 1996. Improved Magnus form approximation of saturation vapor pressure. Journal of Applied Meteorology and Climatology, 35(4), pp. 601-609 DOI: 10.1175/1520-0450(1996)035<0601:IMFAOS>2.0.CO;2.
## Contributors
<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->
<!-- prettier-ignore-start -->
<!-- markdownlint-disable -->
All contributions to this project are gratefully acknowledged using the [`allcontributors` package](https://github.com/ropensci/allcontributors) following the [all-contributors](https://allcontributors.org) specification. Contributions of any kind are welcome!
### Code
<table>
<tr>
<td align="center">
<a href="https://github.com/adamhsparks">
<img src="https://avatars.githubusercontent.com/u/3195906?v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/commits?author=adamhsparks">adamhsparks</a>
</td>
<td align="center">
<a href="https://github.com/cboettig">
<img src="https://avatars.githubusercontent.com/u/222586?v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/commits?author=cboettig">cboettig</a>
</td>
<td align="center">
<a href="https://github.com/karthik">
<img src="https://avatars.githubusercontent.com/u/138494?v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/commits?author=karthik">karthik</a>
</td>
<td align="center">
<a href="https://github.com/noamross">
<img src="https://avatars.githubusercontent.com/u/571752?v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/commits?author=noamross">noamross</a>
</td>
</tr>
</table>
### Issue Authors
<table>
<tr>
<td align="center">
<a href="https://github.com/pat-s">
<img src="https://avatars.githubusercontent.com/u/8430564?u=ca18dea8eda7c5f02b5a94f6242c91ead54d4e83&v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3Apat-s">pat-s</a>
</td>
<td align="center">
<a href="https://github.com/jeffreyhanson">
<img src="https://avatars.githubusercontent.com/u/3610005?u=744bff51a195b78617dd38c203c853c4a87bf37b&v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3Ajeffreyhanson">jeffreyhanson</a>
</td>
<td align="center">
<a href="https://github.com/tosinaregbs">
<img src="https://avatars.githubusercontent.com/u/23707719?v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3Atosinaregbs">tosinaregbs</a>
</td>
<td align="center">
<a href="https://github.com/pssguy">
<img src="https://avatars.githubusercontent.com/u/2775918?u=e1ff2dccc9d53896d99e8ba2865f868589e1ce20&v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3Apssguy">pssguy</a>
</td>
<td align="center">
<a href="https://github.com/amshew">
<img src="https://avatars.githubusercontent.com/u/18128834?v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3Aamshew">amshew</a>
</td>
<td align="center">
<a href="https://github.com/sachserf">
<img src="https://avatars.githubusercontent.com/u/7515749?u=81006c6527a761b8297e5197508c9a459cab406c&v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3Asachserf">sachserf</a>
</td>
<td align="center">
<a href="https://github.com/emdelponte">
<img src="https://avatars.githubusercontent.com/u/16880798?u=a406d1d934266191c84bc6f67505c77edb131e26&v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3Aemdelponte">emdelponte</a>
</td>
</tr>
<tr>
<td align="center">
<a href="https://github.com/choisy">
<img src="https://avatars.githubusercontent.com/u/12168614?u=688c29c3c45cb4296662c2127d70e5d2bd61b128&v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3Achoisy">choisy</a>
</td>
<td align="center">
<a href="https://github.com/sckott">
<img src="https://avatars.githubusercontent.com/u/577668?u=c54eb1ce08ff22365e094559a109a12437bdca40&v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3Asckott">sckott</a>
</td>
<td align="center">
<a href="https://github.com/EFreiberger">
<img src="https://avatars.githubusercontent.com/u/46746047?v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3AEFreiberger">EFreiberger</a>
</td>
<td align="center">
<a href="https://github.com/ssomani10">
<img src="https://avatars.githubusercontent.com/u/8172915?v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3Assomani10">ssomani10</a>
</td>
<td align="center">
<a href="https://github.com/taraskaduk">
<img src="https://avatars.githubusercontent.com/u/16126183?u=ea7bcaa7bc76f572feb793f027c79f14d9894d6c&v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3Ataraskaduk">taraskaduk</a>
</td>
<td align="center">
<a href="https://github.com/oxalisids">
<img src="https://avatars.githubusercontent.com/u/55461047?v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3Aoxalisids">oxalisids</a>
</td>
<td align="center">
<a href="https://github.com/rogarui">
<img src="https://avatars.githubusercontent.com/u/70268591?u=9a889ea593228551c73fd489924f8aa69a180b68&v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3Arogarui">rogarui</a>
</td>
</tr>
<tr>
<td align="center">
<a href="https://github.com/dvictori">
<img src="https://avatars.githubusercontent.com/u/5786553?v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3Advictori">dvictori</a>
</td>
<td align="center">
<a href="https://github.com/hubbarsc">
<img src="https://avatars.githubusercontent.com/u/113626173?v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3Ahubbarsc">hubbarsc</a>
</td>
<td align="center">
<a href="https://github.com/meixilin">
<img src="https://avatars.githubusercontent.com/u/22927483?v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3Ameixilin">meixilin</a>
</td>
<td align="center">
<a href="https://github.com/barryrowlingson">
<img src="https://avatars.githubusercontent.com/u/888980?v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3Abarryrowlingson">barryrowlingson</a>
</td>
<td align="center">
<a href="https://github.com/femiguez">
<img src="https://avatars.githubusercontent.com/u/10869358?u=16370275dff8ff0e73e5b47be18e35fd6cbb89b0&v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3Afemiguez">femiguez</a>
</td>
<td align="center">
<a href="https://github.com/r-3141592-pi">
<img src="https://avatars.githubusercontent.com/u/129575739?v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3Ar-3141592-pi">r-3141592-pi</a>
</td>
<td align="center">
<a href="https://github.com/moyourao">
<img src="https://avatars.githubusercontent.com/u/57421488?u=91fab0f65648a6b529d23e0051de5a17b89abfb5&v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+author%3Amoyourao">moyourao</a>
</td>
</tr>
</table>
### Issue Contributors
<table>
<tr>
<td align="center">
<a href="https://github.com/amine67">
<img src="https://avatars.githubusercontent.com/u/10211999?v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+commenter%3Aamine67">amine67</a>
</td>
<td align="center">
<a href="https://github.com/ckumarMT">
<img src="https://avatars.githubusercontent.com/u/25604303?v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+commenter%3AckumarMT">ckumarMT</a>
</td>
<td align="center">
<a href="https://github.com/GwenGitHub">
<img src="https://avatars.githubusercontent.com/u/20353847?u=0a391b33dc0f0e9cb2c50074b58e2767273e7ec6&v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+commenter%3AGwenGitHub">GwenGitHub</a>
</td>
<td align="center">
<a href="https://github.com/thengl">
<img src="https://avatars.githubusercontent.com/u/640722?u=e1e396e7236035fadf533e47be378e8a51ffbb2e&v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+commenter%3Athengl">thengl</a>
</td>
<td align="center">
<a href="https://github.com/lukuxus">
<img src="https://avatars.githubusercontent.com/u/23721377?u=b69b6b30b439754b1794eb67e881c3e84b2fac49&v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+commenter%3Alukuxus">lukuxus</a>
</td>
<td align="center">
<a href="https://github.com/rsmiller74">
<img src="https://avatars.githubusercontent.com/u/7727644?v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+commenter%3Arsmiller74">rsmiller74</a>
</td>
<td align="center">
<a href="https://github.com/fdetsch">
<img src="https://avatars.githubusercontent.com/u/3940660?u=4318f45d37551fdb2f498abe465de7bf66d65e08&v=4" width="100px;" alt=""/>
</a><br>
<a href="https://github.com/ropensci/GSODR/issues?q=is%3Aissue+commenter%3Afdetsch">fdetsch</a>
</td>
</tr>
</table>
<!-- markdownlint-enable -->
<!-- prettier-ignore-end -->
<!-- ALL-CONTRIBUTORS-LIST:END -->