Archive for April, 2014

new UI design for the OSM data export tool from HOT.


new UI design for viewing an existing OSM export from HOT.


This past weekend I attended State of the Map U.S. (SOTM) in Washington D.C., the annual conference for OpenStreetMap (OSM) users in the United States. Following the conference a sprint day was held at the American Red Cross where conference participants volunteered their time and skills to improve various projects relating to OSM and well as contributing to OSM itself. The project I ended up working on was for the Humanitarian OpenStreetMap Team (HOT), a non-profit organization that assists NGO’s and other groups doing disaster relief, vulnerability analysis and other related humanitarian aid work in developing countries. HOT will typically either have OSM users add data to an un-mapped area in OSM or have locals map the area themselves using GPS and Field Papers.

A recent example of a HOT task was mapping the town of Macenta in Guinea to help doctors and aid workers there stop an outbreak of Ebola. I’ve captured some screen shots of Macenta on both Google Maps and OSM to demonstrate the amount of data contributed by OSM users, Google Maps is on the left and OSM is on the right.

macenta-guinea-ebola-google macenta-guinea-ebola-osm

It’s a bit hard to tell from the zoom level but if you look carefully you’ll see hundreds of buildings, minor roads and some land use areas that have been mapped on OSM which are missing from Google Maps. This was accomplished in a vary short amount of time after HOT made an announcement to the OSM community. Additionally MapBox helped spread the word and has made this process easier with their recently designed user-friendly OSM editor, iD. This data is very important for the doctors and aid workers on the ground to find out where dense population centers are located and to determine how the outbreak is spreading. Nice work OSM contributors and HOT!

One crucial aspect about OSM is that all of it’s data (for the entire world) is available for download for free. The problem is that most services that offer “extracts” of the OSM Planet file (about 400 GB in size so you don’t want to download it all at once!) filter out custom tags that HOT organizations use which are crucial to their work. HOT therefore offers it’s own OSM data extract tool, but the problem with it is that it was sort of thrown together by a developer without much thought to usability. So during the sprint day at SOTM I partnered up with Jue Yang, a talented front-end designer at Development Seed, to redesign the export tool’s interface. The wireframes at the top of this post are what we came up with. We submitted our work as an issue in the HOT Export Tool repository on GitHub. I’m including screenshots of the old version below for comparison.


The original / existing homepage of HOT’s export tool.


What the UI currently looks like when creating a new export.


entering text for fields and selecting an area to export.


The following page where the user selects parameters for their data extract. We simplified this process by combining it with the first step in the new UI design.


What an existing job / export looks like currently. Jue redesigned this part which is the second image at the top of this post.


This is a Processing Sketch that visualizes data relating to climate change from the World Bank Data Indicators. Two visualizations can be toggled using a switch in the upper right corner. Specific data for each country can be viewed by hovering the mouse over each circle. Color scheme was borrowed from Cynthia Brewer’s Color Brewer tool.


  1. I downloaded data from World Bank Data indicators in the Climate Change category. I noticed problems with the raw data:
    • Not every country listed has data for the same year.
    • There are aggregates of countries within the dataset such as “East Asia and the Pacific” or “Sub-Saharan Africa”. While these may be useful, for my data-viz I’m attempting to compare only countries.
  2. To sort out countries only from the raw data:
    • Grabbed category data for 2010 as this was the latest year available for the CO2 kt category.
    • Combined the data into one spread sheet.
    • Used a join in QGIS to a world dataset file so that only matching countries would be imported. I used the ‘Country Code’ key from the World Bank data and joined it to Natural Earth‘s ne_10m_admin_0_countries.shp ‘Adm0_a3_is’ column.
    • To only keep matching countries from Natural Earth and the World Bank I filtered out data with NULL values using an SQL WHERE clause:
      • SQL SELECT * FROM ne_10m_admin_0_countries WHERE "pop_gni_co2_2010_Country Name" != 'NULL' AND "pop_gni_co2_2010_GNI_2010" !='' AND "pop_gni_co2_2010_CO2_2010" != '' AND "pop_gni_co2_2010_POP_2010" != ''
  3. I noticed that some countries lacked data for 2010. To fix this I manually added data from the next available previous year to try to fill in the gaps (probably not the most statistically sound method). These problem countries are listed below. A couple countries such as Myanmar and North Korea did not have any data from the World Bank so were excluded from the viz.
    • Countries shown that had no data for 2010. Their values were replaced with data from next available previous year. The countries and year of data is listed below:
      • Argentina GNI (country & per capita) 2006
      • Greenland GNI (country & per capita) 2009
      • Libya GNI 2009
      • Sudan and South Sudan: I split Sudan’s CO2 evenly as South Sudan has no data in this category (probably due to it being a relatively new country)
      • Djibouti GNI (country & per capita) 2005
      • Somalia / Somaliland GNI (both) 1990
      • Iran (both GNI) 2009
  1. In future iterations the following features would be added:
    • Animate the circles when switching visualizations.
    • Add an ability to turn on/off the logarithm.
    • Highlight all circles by region when hovering over a region in the legend.
    • Improve the typography.
    • Add a label for the toggle switch.
    • Show a projection of the trend into the future
    • Toggle a view to display the data on a world map.

Code is available on Github.


So after a lot of work I’ve arrived at the second major update for my Major Studio Two project which tells the story of my brother Mike’s thru-hike on the Pacific Crest National Scenic Trail through an interactive web media experience.

Major milestones include:

1. Debugging a Leaflet map interaction where the animated marker (representing my brother’s position) triggers popups to open on the stationary markers.

2. Improving the interaction between scrolling and the animated marker start and stop using the JQuery Waypoint library. Now when the user scrolls, then animated marker will only start and stop when the user passes the correct chapter chronologically. I found that users tend to scroll up and down through the site fairly quickly to get an idea of long it is. Previously this would have caused the animated marker interaction to be buggy.

3. I’ve also managed to do some user testing for the site today which has proven to be super helpful. I will admit that all the folks that have helped me out have been DT students so I will also test the site with some people outside of my program for a different perspective. Here’s some feedback I’ve received so far:

1. The title bar doesn’t stand out enough and blends in with the rest of the site too much. The url of the site would also help with the title (no url as of yet…)

2. The Alan Watts quote on the opening slide is hard to read; try masking the area behind it with a transparent fill. (fixed)

3. Initially the user has no idea who the main character is. It would be helpful to add some background about Mike and the PCT in general prior to the start of his story.

4. The map transition from overview to the start of the trail should be slightly smoother and more seamless.

5. It’s somewhat confusing where the marker is going in relation to the content. Some suggestions to improve this are to break up each chapter’s path so that the marker will travel shorter distances. Then have photos or other text blocks trigger the marker to move. This would also help connect the particulars of the content with the map. Text blocks could even be highlighted so that when the user hovers or clicks on them their location is displayed in the map. Another suggestion was to repeat each chapter title over the top of the map so that the user is aware of the connection between the area being traveled and the map chapter.

6. When the marker is moving the map tends to shake. I’m not sure if this is something I have a lot of control over but will definitely think about.

7. Add a locator map to show the regional overview (ie: the current state or entire trail) so that the user has an idea of where on the whole trail Mike is.

8. Use Lazy Loading JQuery library to load the images as the user scrolls. This will help cut down on overall loading time and improve performance of the site.

9. Possibly consider adding menu options to the title bar so that the user can jump to certain chapters. This will be somewhat tricky to implement but may be necessary. Also adding forwards and backwards arrows could be helpful with navigating chapters. Worth testing though.

Positive feedback I recieved was that the site is very readable and would make a good narrative template overall. The first image is liked by everyone I’ve tested and serves as a good introductory slide.

Lastly, some updates from a house keeping perspective are to make the site cross-browser compatible (right now I’ve only been developing it using Google’s Chrome browser) and organizing the javascript code into separate modules / files.

Looks like some more updates and then another round of user testing will be needed. Looking forward to completing this project and submitting it to the student mapping competition for NACIS this year in Pittsburg, PA.