Wednesday, 12 August 2015

OSM Retail Survey: Conclusions-1

OSM has thrived by bringing together a community with diverse interests, and aligning their efforts behind a common purpose. In thinking how best to improve retail coverage it seems useful to consider how different groups with different interests and different skills will be able to contribute.

The most obvious question for the community is how the existing tools might be improved. But I am not going to start there. Instead I will begin with how contributors might view the priorities - because that will determine which tools will be of greatest help.

My starting point is based on findings from the survey:
  • In a some localised areas retail data in OSM is the most comprehensive retail data that is generally available. Because OSM data has a degree of structure it should be capable of supporting certain types of structured search that are extremely difficult to achieve in any other way. These are the areas where the data will be of greatest value to end users and hence of greatest interest to application providers
  • We are still a long way from being able to offer comprehensive retail data across the whole of the UK. In the foreseeable future this means that most viable applications based on OSM data are likely to have a local focus, rather than aiming for national coverage. So far only a few areas have been really thoroughly mapped. One priority is to increase the number of thoroughly mapped areas.
  • Elsewhere, whatever issues data users find with the consistency and accuracy of UK retail data in OSM, the impact of those issues is small in comparison to the amount of retail data that is missing from OSM. Another priority is to reduce the volume of missing retail data.
To address missing data, I assume the community needs to expand the number of contributors, as well as encouraging existing contributors to add more basic retail data. We need to ensure that the process of collecting and contributing data is both satisfying and productive.

Most contributors have only a limited choice of where to map. The question we need to help them with is how to make the biggest impact in their local area. Some contributors have more choice of where they map. The question we need to help them with is where they can make the biggest impact.

If OSM is going to provide a decent platform for viable applications based on retail data, then the priority is to bring more areas of the UK up to a standard that compares with the best. OSM data doesn't have to be complete in order to be the best available source of retail data in a well-defined area: but it should be getting near complete. In towns and smaller cities individual contributors can quickly make an impact, by bringing retail data up to a good standard across a well-defined area. In an ideal world they might chose a location that would most interest potential application providers – perhaps a university town, or a city that attracts a large number of visitors.

I'd like to think that contributors who want to improve retail data will start by assessing how retail coverage currently stands in their chosen area. For a rough idea, they can examine the standard map, or for more precision they can compare the number of shops in the OSM database with an estimate of how many there should be. There are various ways to get that estimate, but that's a separate question which I'll defer for now.
  • If local coverage is currently under 25%, then this part of the map is still close to being a blank canvas. OSM data is far from providing the best source of retail data, and there will still be gaps in some of the most commonly mapped features, such as post-offices and pubs. The first priority is to make a start, develop technique, and demonstrate progress to encourage others. For a contributor's own motivation, they should begin with whatever interests them personally. This probably includes retail outlets that they are familiar with (i.e. ones that their family, friends and neighbours use regularly). Beyond that, major retail premises, such as supermarkets, banks and larger high street stores are relatively easy to tag, and are all properly rendered on the standard map. Their relatively large scale helps to build visibility. To help raise awareness add any retailers with a high public profile. This could include any well-known local specialists, those who advertise heavily, those who regularly feature in the local paper, or take an active part in the local chamber of trade.
  • If local coverage is around 25-50% then a fair number of shops will appear on the standard map, but there will still be plenty that are missing. Quite often some retail categories will have been well covered (pubs often seem to appear first), while others still have to be added. The priority now is to build momentum. The quickest results will be achieved in densely occupied retail zones such as the central shopping area and larger retail parks. Complete coverage is still some way off, and trying to include everything at this stage will slow things down. It is more important to include major outlets, and a representative sample of outlets that are of high public utility, and widespread interest. It seems to me that this should include retailers that cater for a broad section of the population – both their daily needs (convenience stores, post offices, pharmacies, take-aways, caf├ęs, pubs), and more significant purchases (electrical goods, clothing, furniture, etc.). Others will have better insight into those catering for specific groups of customer (visitors, students, etc.), and in some towns these groups will be particularly important.
  • Once local coverage reaches around 50-75% then OSM data is providing some of the most complete retail data that is generally available. The standard map will contain a good number of shops – particularly in the town centre. But anyone familiar with the area will still find it fairly easy to spot shops that are missing. Particularly outside the main shopping areas there will be shops scattered across residential and commercial areas that haven't been added.  Now is the time for contributors to work towards something approaching complete coverage. Missing shops are likely to be the more specialised, smaller and more quirky independent retailers, shops outside the central retail core, in suburban shopping parades, and corner shops in residential areas.
  • Over 75% coverage means that locally OSM is capable of providing some of the most comprehensive retail data that is generally available. Contributors will find it increasingly time-consuming to deal with the remaining gaps, and the more difficult categories are the most likely to have been left aside. Now is the time to include them. This is also the time to verify that existing data is up to date and consistent. There will be opportunities to add value with information that will be of use to different types of data user. This might include features such as wheelchair access, ATMs, non-standard opening times, specialist services, etc. Beyond this, contributors have  choice. They can continue to add unreasonable levels of detail, that will never be used. Or better, broaden the scope of their survey into neighbouring towns and villages.

If this is broadly how things work, then I'd suggest the following priorities to help contributors:

  • Firstly, contributors are encouraged by seeing the results of their work. Currently the standard map is the main source of such feedback, but it doesn't show all retail outlets, and it doesn't render all retail characteristics that contributors add. It is unrealistic to expect the standard map to render everything, so I'd like to see a different way of showing contributors the results of their work: one that doesn't depend on adding increasing detail to the standard map.
  • Secondly, for areas where retail data is relatively thin, contributors who have some choice of where to map may benefit from guidance on where and how they can make the biggest impact. Tools that highlight areas where there is a substantial amount of missing retail data could save them time. Suggesting areas where retail data could be of high utility may influence their choice of where to map.
  • Thirdly, retail data generally has to be collected by survey, and there is a lot missing. Tools that help contributors collect data in the field (i.e. on the high street) will make the process more satisfying and help to speed the process
  • And finally, once  retail data in an area is relatively complete, the emphasis will change from improving coverage to improving consistency and adding value. Bulk edits won't help much, but tools that highlight inconsistencies and quirks in retail data will help contributors identify issues and improve quality.

1 comment:

Riddari said...

I have an idea for the maintenance aspect. A seperate field to add to retails which would be contact info for the business, let's call it email_maintainer. Once a year an automated system sends out a mail to each email_maintainer (one could be responsible for tens of branches) with information on each listing and providing a link to fix each.

For example it would say that listed opening hours are 10-11 but they are in fact now 9-11. By clicking on the link the user goes to a webpage where the information appears to him in a friendly manner, allowing him to modify the value and submit. The data would be changed in the database, possibly under user OSM Retail Fixes

Each link would be generated with some rules so that only the exact link works to update, you can not guess your way.

The retail maintainer could also flag the business as being closed or moved, that could create a Note so someone can then fix it.

We have been thinking of doing this in Iceland, it is a fairly small country in term of population and if it works great then maybe this could be expanded.

--
OSM user Stalfur