Back in 2011, ESRI released Story Maps: a series of applications that allow you to use ArcGIS Online web maps to share a story in an interactive webpage.
Last year they began promoting it to a wider audience and so I was keen to get involved. Originally I had planned to produce something last summer between the ESRI International User Conference and the International Cartographic Conference, but better late than never I suppose.
My aim / idea
To load a planned itinerary onto a standard in-car satellite navigation system (satnav), investigate its usefulness in finding my way around California and investigate the value of the data captured by the satnav in documenting the trip. Oh, and to present it as an ESRI Story Map.
Hire car with satnav (I opted to test a traditional Garmin rather than a newer tablet version)
Laptop (or other portable) computer
ESRI ArcGIS Online (Using a free global account)
Table editing software (e.g. Microsoft Excel)
Web host (I used my blogsite on Windows Azure)
Web mapping software suitable for planning (e.g. Bing Maps)
Step 1 – Planning
So I used Bing Maps because it combines a search engine with locations of attractions. This makes it quick and easy to discover what attractions are in the area and to save them to your itinerary. Although I personally prefer Google as a search engine and Google Maps as a web map, this planning task just seems to me to be a lot easier in Microsoft’s Bing Maps. I’ve covered this at the end of a previous post but one tip is that you can save the pop-ups/pushpins from the search results straight into your itinerary. Save the trip out as a GPX file on the laptop.
Step 2 – Loading
This bit is quite awkward. You’ve waited ages to get the bus to the car hire depot, you’ve queued in line, you’ve told the man at the desk at least ten times that had you wanted to hire a damn convertible sports car you would have done so on your original booking rather than selecting economy, it has dawned on you that the cost breakdown of the sat nav hire is probably more than the US retail price in Wal-Mart, you have finally been shown which cars you can choose from and you’ve selected and inspected the one that you want. So then all the staff stare at you in bewilderment as you stay sat in the parking lot and start opening up your laptop! Sure you’re running three hours late and need to get to the hotel before dark but trust me, this is going to save you so much time and hassle on the rest of the trip.
The next bit depends on manufacturer but in the case of Garmin, you may have read that you need to download and install software such as the Garmin POI Loader or Garmin Communicator Plugin or Garmin Connect or Garmin Basecamp. Well the good news is that you don’t need any of these. The process is far simpler:
Using the lead that comes with the satnav (or often a standard USB to MicroUSB, that you may well already have if you own a portable hard disk drive) simply plug the satnav into your laptop. Once recognised, you can treat is like an external hard disk and save the GPX file onto it in an existing folder called ‘Favourites’, or I guess in the States it would be Favorites.
Turn off and disconnect the satnav, then turn it on again (without the laptop). Navigate to the Favourites folder (not the Points of Interest folder) and you will see you itinerary as a list of points of interest. Now, rather than messing about everyday with searches, addresses or postcodes, simply click on the next place in your favourites list and you’re on your way. I cannot overstate how useful this was when trying to see many places and cover large distances over a short space of time. Waking up in the morning, having time to eat breakfast, knowing that I could just get into the car and go was blissful.
I guess with the newer tablet versions, searching by internet might make pre-planning less valuable, but who is to say you’ll always have a working internet connection.
Step 3 – Collecting
Well this step is the easiest because the satnav will do it all for you. Wherever you drive and wherever you stop, the satnav will log it all for you. You just need to drive the car and remember to keep the satnav turned on whenever you’re on the move.
Step 4 – Downloading
Slightly awkward again, for instance if you try to do this at airport car returns, so I decided to do this before returning the car (the compromise is that you therefore sacrifice that last little bit of data).
So before you give the car and the satnav back to the hire company, plug it back into your laptop and you’ll find a folder of log files. On the Garmin Nüvi model I was using, this folder was called ‘Archive’. Within it was a set of numbered log files. Copy them all across to the laptop to be sure, but the likelihood is that you’ll only need the most recent one or two. They tend to overlap considerably, you can think of them as saves. If you don’t want the rental to know where you’ve been then delete them before you give the satnav back 🙂
Step 5 – Viewing the raw data
Firstly I wanted to see if the captured GPX data could be loaded into a web map service without any data preparation and how it would compare against the original data exported as GPX from a web map service (in my case Bing Maps).
Log in to ArcGIS online at arcgis.com. If you are new then you’ll first need to create an account. I used a free account to achieve all of this. Go to create map and then add layer from file.
Choose your GPX file and voila!
I was pleasantly surprised to see that this raw data is actually very well structured and loads into ArcGIS online just fine. You do get two lots of data however, the waypoints and the tracks. The best definition I found was in the OpenStreetMap literature in reference to one of the original Nuvi models.
The tag structure is different to the GPX plan that came from Bing and because you have all of the points captured along the route, including extra tags for speed and elevation, then the Garmin file can seem quite daunting. But the time stamps are useful markers and the concept is obviously the same as they both adhere to the same standards.
If you wish, you can load and view the planned itinerary data in the same way.
Step 6 – Data cleaning
What we currently have is raw GPX data from the satnav and from Bing Maps that is perfectly suitable for ArcGIS online. However, for the Countdown Story Map application we are going to wish to add extra attribution to the data, e.g. post-trip comments and photos, and so we will need to manipulate the Bing Maps (planned itinerary) data slightly.
At first I tried adding this extra data directly into the GPX file in a code editor (e.g. Sublime Text) by using the unused tags available in the GPX file format. Whilst there were enough free tags, on testing it became apparent that the countdown app is heavily coded to depend upon the fields of rank and level which are not valid GPX tags. I strongly advise you make your data match these field names and not the other way round!
Rank is the order in which the locations will appear in the story map. It made sense therefore for my trip to add a new column for this and to use the order in which I visited them.
Level is the ‘zoom level’ of the ArcGIS web map service that you wish that location to be viewed at. These can be calculated by trial and error once the app has been created and is ready for testing. 0 is the smallest scale (the most zoomed out) and 19 is the largest scale of the service (the most zoomed in).
To add all this attribution, the simplest way is to open your GPX up as a table in a GIS or spreadsheet package. I opened it directly into Microsoft Excel without any problems. I added the rank and level columns (the levels you can always change later) and I added post-trip comments and hyperlinks (urls) to images and their thumbnails. The fields you are adding are essentially populating a panel that will use your ArcGIS online pop-up if you have any active in your overview map.
Save your table out as a CSV file.
At this stage, you could also tidy up the Garmin GPX file, but I chose to just turn off the unrequired data in ArcGIS online (see step 7, below).
Step 7 – Map creation in ArcGIS online
As before, log in to ArcGIS online and choose to create a new map. If you are not yet in a map window, go to ‘Open in ArcGIS.com map viewer’. This map will be our overview map containing both the route data captured by the satnav, and the planned itinerary.
If it doesn’t do so automatically, select ‘Content’ in the panel on the left of the screen. Next, go to ‘Add’ then ‘Add layer from file’ and add the Garmin GPX file.
Choose a suitable base map for the overview. I used the National Geographic map, I think it is the nicest within the ArcGIS options but you can load in others if you have permission and the URL.
Left-click on the satnav data’s layer name and you’ll see the time-stamped data. Uncheck all of the data that has ‘Trackpoints’ in the name. This will remove the track data, which is a dense series of satellite recorded points tracking along our route, and will leave us with just the wayfinder data which is essentially one line for each leg of the journey. As it is less dense, it will be easier to work with than the points.
Now we can style these lines. Each line (or leg) will have a separate name, for example ACTIVE LOG: 17 JUL 2013 16:53. Go to each of these layers in turn, and click on the little drop-down arrow to the right of the layer name and select ‘Change Symbols’. Choose a single symbol and click on ‘Change Symbol’. You will then be presented with a nice, easy, intuitive user interface from which you can style your route line.
Click on the top level layer name once more to collapse it back to how it was before.
Next add the plan or itinerary, but this time import the new CSV file instead of the GPX file. This will have added to our map, the points for and any notes of, the locations we had planned to visit.
Once more we can style the data by clicking on the dropdown arrow to the right of the layer name and choosing ‘Change Symbols’. Once the symbols have been chosen, go to the same dropdown menu and select ‘Configure Pop-Up’.
This will allow us to configure the pop up box that appears when a user clicks on the point to interact with the map. ESRI have helped by pre-populating the options with a default setting but change them as you wish. Choose to ‘Save Pop-up’. Then simply click on any of your points to see how they look. If you don’t like it, then try again!
The last thing to do here, is to save the map and make it public. At the top of the screen you will see a save button with a dropdown. Choose ‘Save as’ and you can set the title and summary that will also appear in the final app (although it can be overridden in the app setup later if you so wish).
Next go to Share, and tick the checkbox for Everyone (public) and click ok. This will allow the app to load the map.
Now create a new map and once more load in the itinerary data. This map will become the detailed view within the app. Once more style the symbols as you wish. This time select a basemap better suited to a detailed view, possibly Imagery for example.
Save this map and also make it public. Finally make a note of the two map id’s. You can find this in the URL of each map or by selecting the maps from the ‘My Content’ page of ArcGIS online.
Step 8 – Obtaining and hosting the app
Some apps can be completely set up from within ArcGIS online. The countdown story map application and all of the other ESRI Story Maps apps can be downloaded either from ESRI or from GitHub as a zipped folder (.zip file).
Next unzip the folder and the application should already be set up as a fully-functioning example. Inside the folder is a file called index.html. This is what is used to run the application. The other file of interest is called config.js which is used to customise the application.
Firstly, we must test that the application works with the sample data. For index.html to run, the entire folder must be accessible online as a URL.
Getting a valid URL proved to be something of a stumbling block for me. I wasn’t keen on trying to deploy an app because I’m not really that savvy on code-based development. Once unzipping the folder, I tried running the index.html file ftp server links and public links on Dropbox but neither of these work, so you need to host it online somewhere. I managed to use FireFTP to transfer it by FTP onto my Windows Azure-based WordPress site and that worked.
Once you have a place online, you just need to work out the URLs. It turns out that this is actually quite straightforward. For example I renamed my folder to CaliMap and loaded it into wwwroot/ within my cloudspace. So the html file was at the filepath wwwroot/CaliMap/index.html. My website url is http://christopherwesson.azurewebsites.net so my html file must have the url http://christopherwesson.azurewebsites.net/CaliMap/index.html. Using the same logic you can deduce the url of every file within the folder. I used this to also host my photos and I put the urls into the data table.
Obviously if you update your data (the CSV file), then you will also need to update your maps in ArcGIS online. I reloaded the data and went through the whole styling process all over again. So can someone please let me know if they find a way to just refresh the data table!
Step 9 – Configuring the app
This step is incredibly simple, so long as you have the rank and level fields as suggested earlier. Open up the config.js file in a code or text editor. ESRI have kindly put instructions in there, or refer to the readme file that was also a part of the app download. Basically the main thing you need to do is to change the two map id’s from the ESRI examples to your own maps. These are the 2 id’s we noted down earlier from ArcGIS online.
Save the config.js, if you worked on it locally then push it back up to the cloud, and that is it.
Use the URL of the index.html file (as discussed in step 8) to test whether your app works.
So here we are; my map of my trip around Southern California and thereabouts.
Examples of what we can see
So what does this map actually show us? Well, obviously it shows what I got to see and what I missed and it shows where I went.
If you click on the itinerary one at a time using the numbers, then you get a pretty good concise summary of my trip. But if you make the overview map the main map, zoom in on the route and start to interrogate it, then actually it shows us far finer detail than I expected.
The map offers me a very graphic illustration of the choices that I had to make, for example between sticking to the coastal road and travelling via Minuteman beach as planned or travelling via Grimes Canyon and then detouring along the main interstate highway to ensure I reached my motel before midnight (as I ended up doing).
Although I am a little obsessive in planning trips online before I go (personally I’d rather do this than walk around with a guidebook once I’m there), there are some things I forgot to account for. For example, where to stop for fuel. I was quite impressed than looking back on the route data, I can actually pinpoint the places I stopped to fill up, generally noticeable because you go slightly off route (as in the extract image above).
You could take this further and include the speed and elevation data. Speed for instance might shed more light on how successful my route and itinerary planning were.
Loading an itinerary into a satnav is a very useful process and one that on long-distance trips I will definitely use again.
ArcGIS online is very good for making quick maps of routes, although I found the base mapping a little underwhelming at certain scales. It is also quite intuitive to use.
ESRI story map apps each require a little understanding but are an excellent way of sharing a story with geographical context. I think there is room for ESRI to improve both their functionality and customisation but it does not at all surprise me that National Geographic for example have worked with ESRI on creating their own custom equivalent for their news articles.
The countdown template I used could be significantly improved if you could set a separate zoom level for each map window.
I am pleased I got there in the end, I feel that this little bit of research has been a worthwhile exercise.