The initial impetus for setting up a river levels website came in the aftermath of flooding in my local area. While the Environment Agency's own website does provide river level information, it has to be said that the presentation of the data is somewhat limited. So I decided to set up my own version.
In particular, what I thought I could do better was to present the data in a more accessible visual format, and with better cross-referencing between different sites (so, for example, when viewing the data for the River Avon at Evesham, there are also quick links to other locations on the Avon and other rivers near Evesham). Putting the locations onto a Google map also makes it easier to find those which are relevant as you don't need to know precise names.
As originally set up, this was all entirely unofficial. The Environment Agency didn't make the necessary data available under a suitable licence, so to get it I needed to screenscrape the EA website. This was less than ideal for a number of reasons, including the fact that it's rather tedious as well as the risk that the EA would try to get the site shut down.
However, things have taken a turn for the better. Following a concerted campaign from Open Data campaigners, the EA is in the process of releasing more of its data under the Open Government Licence, and that now includes river level and flood warning data.
To begin with, rather than publish the data in a suitable format itself, the EA contracted it out to a commercial partner, Shoothill Ltd. This enabled a rapid rollout of the data and made it available for the first time in a re-usable format.
However, there were (and are) some issues with the Shoothill data. A key one is that it doesn't quite match the data published on the EA's own website. There are also some inconsistencies, such as some tidal and downstream locations not being correctly identified.
A more long-term issue with the Shootill data is that, although initially available at no cost, it isn't Open Data and therefore is subject to their own licensing terms.
More recently (Spring 2017), the EA has finally started publishing its data directly via its own API. There are still a few teething problems with this, notably some incorrect and missing data, and the service is currently still in beta at the time of writing. However, it does, finally, make the raw EA data available to developers without an intermediary and published under a licence that enables full re-use.
Taking advantage of the new EA data has meant completely re-engineering the underlying platform on which the website runs. As well as now getting data directly from the EA rather than via Shoothill, we now also need to cope with the fact that the environment is a devolved function and hence data for Wales and Scotland is supplied by their own agencies rather than the Environment Agency in England. So there are now three separate sources of data, which are not entirely consistent. This needs to be merged into a reasonably unified single dataset for presentation on the website.
Although these changes could have been made entirely in the background, I've also made the decision to update the presentation of the website along with them. Most of these changes are relatively minor, and, as far as possible, the existing URL structure has been retained so that bookmakrks and deep links will (mostly!) continue to work.
The biggest change is the replacement of the region and catchment based navigation hierarchy with one based on geographic counties. In one sense this has been forced by the fact that the new sources of data don't reliably include a catchment name, whereas the county can always be determined from the latitude and longitude of the measuring station. But, on the other hand, counties make more sense from a human perspective, as nearly everybody knows which county they are in but may not be familiar with Environment Agency regions and river basin catchment areas.
The new site is still a work in progress, and further tweaks will be made in the light of experience.