new UI design for the OSM data export tool from HOT.

hot_export_ui_revision

new UI design for viewing an existing OSM export from HOT.

hot_export_ui_revision2

This past weekend I attended State of the Map U.S. (SOTM) in Washington D.C., the annual conference for OpenStreetMap (OSM) users in the United States. Following the conference a sprint day was held at the American Red Cross where conference participants volunteered their time and skills to improve various projects relating to OSM and well as contributing to OSM itself. The project I ended up working on was for the Humanitarian OpenStreetMap Team (HOT), a non-profit organization that assists NGO’s and other groups doing disaster relief, vulnerability analysis and other related humanitarian aid work in developing countries. HOT will typically either have OSM users add data to an un-mapped area in OSM or have locals map the area themselves using GPS and Field Papers.

A recent example of a HOT task was mapping the town of Macenta in Guinea to help doctors and aid workers there stop an outbreak of Ebola. I’ve captured some screen shots of Macenta on both Google Maps and OSM to demonstrate the amount of data contributed by OSM users, Google Maps is on the left and OSM is on the right.

macenta-guinea-ebola-google macenta-guinea-ebola-osm

It’s a bit hard to tell from the zoom level but if you look carefully you’ll see hundreds of buildings, minor roads and some land use areas that have been mapped on OSM which are missing from Google Maps. This was accomplished in a vary short amount of time after HOT made an announcement to the OSM community. Additionally MapBox helped spread the word and has made this process easier with their recently designed user-friendly OSM editor, iD. This data is very important for the doctors and aid workers on the ground to find out where dense population centers are located and to determine how the outbreak is spreading. Nice work OSM contributors and HOT!

One crucial aspect about OSM is that all of it’s data (for the entire world) is available for download for free. The problem is that most services that offer “extracts” of the OSM Planet file (about 400 GB in size so you don’t want to download it all at once!) filter out custom tags that HOT organizations use which are crucial to their work. HOT therefore offers it’s own OSM data extract tool, but the problem with it is that it was sort of thrown together by a developer without much thought to usability. So during the sprint day at SOTM I partnered up with Jue Yang, a talented front-end designer at Development Seed, to redesign the export tool’s interface. The wireframes at the top of this post are what we came up with. We submitted our work as an issue in the HOT Export Tool repository on GitHub. I’m including screenshots of the old version below for comparison.

HOT-original1

The original / existing homepage of HOT’s export tool.

HOT-original2

What the UI currently looks like when creating a new export.

HOT-original3

entering text for fields and selecting an area to export.

HOT-original4

The following page where the user selects parameters for their data extract. We simplified this process by combining it with the first step in the new UI design.

HOT-original5

What an existing job / export looks like currently. Jue redesigned this part which is the second image at the top of this post.

scatterplot-v8-combined

This is a Processing Sketch that visualizes data relating to climate change from the World Bank Data Indicators. Two visualizations can be toggled using a switch in the upper right corner. Specific data for each country can be viewed by hovering the mouse over each circle. Color scheme was borrowed from Cynthia Brewer’s Color Brewer tool.

Process:

  1. I downloaded data from World Bank Data indicators in the Climate Change category. I noticed problems with the raw data:
    • Not every country listed has data for the same year.
    • There are aggregates of countries within the dataset such as “East Asia and the Pacific” or “Sub-Saharan Africa”. While these may be useful, for my data-viz I’m attempting to compare only countries.
  2. To sort out countries only from the raw data:
    • Grabbed category data for 2010 as this was the latest year available for the CO2 kt category.
    • Combined the data into one spread sheet.
    • Used a join in QGIS to a world dataset file so that only matching countries would be imported. I used the ‘Country Code’ key from the World Bank data and joined it to Natural Earth‘s ne_10m_admin_0_countries.shp ‘Adm0_a3_is’ column.
    • To only keep matching countries from Natural Earth and the World Bank I filtered out data with NULL values using an SQL WHERE clause:
      • SQL SELECT * FROM ne_10m_admin_0_countries WHERE "pop_gni_co2_2010_Country Name" != 'NULL' AND "pop_gni_co2_2010_GNI_2010" !='' AND "pop_gni_co2_2010_CO2_2010" != '' AND "pop_gni_co2_2010_POP_2010" != ''
  3. I noticed that some countries lacked data for 2010. To fix this I manually added data from the next available previous year to try to fill in the gaps (probably not the most statistically sound method). These problem countries are listed below. A couple countries such as Myanmar and North Korea did not have any data from the World Bank so were excluded from the viz.
    • Countries shown that had no data for 2010. Their values were replaced with data from next available previous year. The countries and year of data is listed below:
      • Argentina GNI (country & per capita) 2006
      • Greenland GNI (country & per capita) 2009
      • Libya GNI 2009
      • Sudan and South Sudan: I split Sudan’s CO2 evenly as South Sudan has no data in this category (probably due to it being a relatively new country)
      • Djibouti GNI (country & per capita) 2005
      • Somalia / Somaliland GNI (both) 1990
      • Iran (both GNI) 2009
  1. In future iterations the following features would be added:
    • Animate the circles when switching visualizations.
    • Add an ability to turn on/off the logarithm.
    • Highlight all circles by region when hovering over a region in the legend.
    • Improve the typography.
    • Add a label for the toggle switch.
    • Show a projection of the trend into the future
    • Toggle a view to display the data on a world map.

Code is available on Github.

 

So after a lot of work I’ve arrived at the second major update for my Major Studio Two project which tells the story of my brother Mike’s thru-hike on the Pacific Crest National Scenic Trail through an interactive web media experience.

Major milestones include:

1. Debugging a Leaflet map interaction where the animated marker (representing my brother’s position) triggers popups to open on the stationary markers.

2. Improving the interaction between scrolling and the animated marker start and stop using the JQuery Waypoint library. Now when the user scrolls, then animated marker will only start and stop when the user passes the correct chapter chronologically. I found that users tend to scroll up and down through the site fairly quickly to get an idea of long it is. Previously this would have caused the animated marker interaction to be buggy.

3. I’ve also managed to do some user testing for the site today which has proven to be super helpful. I will admit that all the folks that have helped me out have been DT students so I will also test the site with some people outside of my program for a different perspective. Here’s some feedback I’ve received so far:

1. The title bar doesn’t stand out enough and blends in with the rest of the site too much. The url of the site would also help with the title (no url as of yet…)

2. The Alan Watts quote on the opening slide is hard to read; try masking the area behind it with a transparent fill. (fixed)

3. Initially the user has no idea who the main character is. It would be helpful to add some background about Mike and the PCT in general prior to the start of his story.

4. The map transition from overview to the start of the trail should be slightly smoother and more seamless.

5. It’s somewhat confusing where the marker is going in relation to the content. Some suggestions to improve this are to break up each chapter’s path so that the marker will travel shorter distances. Then have photos or other text blocks trigger the marker to move. This would also help connect the particulars of the content with the map. Text blocks could even be highlighted so that when the user hovers or clicks on them their location is displayed in the map. Another suggestion was to repeat each chapter title over the top of the map so that the user is aware of the connection between the area being traveled and the map chapter.

6. When the marker is moving the map tends to shake. I’m not sure if this is something I have a lot of control over but will definitely think about.

7. Add a locator map to show the regional overview (ie: the current state or entire trail) so that the user has an idea of where on the whole trail Mike is.

8. Use Lazy Loading JQuery library to load the images as the user scrolls. This will help cut down on overall loading time and improve performance of the site.

9. Possibly consider adding menu options to the title bar so that the user can jump to certain chapters. This will be somewhat tricky to implement but may be necessary. Also adding forwards and backwards arrows could be helpful with navigating chapters. Worth testing though.

Positive feedback I recieved was that the site is very readable and would make a good narrative template overall. The first image is liked by everyone I’ve tested and serves as a good introductory slide.

Lastly, some updates from a house keeping perspective are to make the site cross-browser compatible (right now I’ve only been developing it using Google’s Chrome browser) and organizing the javascript code into separate modules / files.

Looks like some more updates and then another round of user testing will be needed. Looking forward to completing this project and submitting it to the student mapping competition for NACIS this year in Pittsburg, PA.

 

Triggering PopUps with an AnimatedMarker in Leaflet JS from Chris Henrick on Vimeo.

With the help of my web-developer guru friend Eric Brelsford, I’ve managed to figure out one of the last pieces of my Map Story project which was having the animated marker on the map open popups on stationary markers that represent points of interests, natural features, towns, etc. I’ve documented the code in a GitHub Gist which can be viewed live on this block. Excited to be wrapping this project up soon (hopefully!).

Last month I presented my Major Studio One final, Exploring Bias in Spatial Data, at GeoNYC, the Meet-Up group for all things relating to maps and geo-spatial technology. (p.s. this video does a better job of explaining this work than the previous blog post I made.)

Go to minute 47 for the start of my presentation, it’s about 5 minutes long:

Geo NYC Meetup – February 2014: Student Showcase from Boundless on Vimeo.

2014-03-14_15-26-01

2014-03-14_15-26-20

2014-03-14_15-26-30

2014-03-14_15-26-48

2014-03-14_15-27-04

For my major studio two class this semester I’m working on creating an interactive web piece that tells the story of my brother Mike’s epic trek across the Pacific Crest National Scenic Trail in 2013. My inspiration for creating this piece came from a story by the New York Times titled The Russia Left Behind which features a map that animates as the user scrolls through the content. I’m attempting a similar feel with my piece, but rather than using the D3 JavaScript library for rendering the map I’ve chosen to use a combination of Leaflet JS, a few plug-ins for Leaflet including the Animated Marker and Maki Markers plug-ins and JQuery Waypoints to tie the scrolling of the content to creating changes in the map. Last of all and perhaps most importantly the base map I’m using is courtesy of Stamen Design‘s Terrain Tiles. Stamen did a superb job of using open data from both the OpenStreetMap Project and USGS to beautifully render topography in North America. I’m taking advantage of this open-source tile set to add an important visual element to my brother’s story; the user has a detailed visual representation of the mountainous landscape my brother hiked through. In fact you can see how this plays out in the story itself when he describes “the climb”, the first of many ascents on his trip, which he mentions in the second chapter (Warner Springs to Idyllwyld).

For the content of this story I’m scraping text from my brother’s blog using a Python script that uses the beautiful soup library. The photos and videos he  has shared directly with me and generously gave me permission to use. Needless to say there’s an added editorial and curated component to this work from choosing the best photos and checking spelling and grammar. Being comfortable with editing writing from my undergraduate studies in Geography and Urban Studies is definitely helping here!

I’m almost set to do some user-testing, with the intention to get some feedback to improve the next iteration of the project. One last part I’d like to accomplish is for the animated marker to trigger pop-ups on the stationary markers as it passes them. I also have the task of editing and curating my brother’s writing which is another huge undertaking given the detail and length of writing from his blog posts.

The project in its current state can be viewed here, though at this time I have only tested it in the Chrome browser so it may not work in Safari, Firefox and Internet Explorer.

IMG_0022

For our first Data Visualization class of the Spring 2014 semester our professor gave us a problem to solve in twenty minutes: visualize the numbers 10, 30 and 60 using materials he provided such as colored cotton balls of various sizes, crayons, popsicle sticks, pipe cleaners, etc. We were to draw inspiration from a blog post by the website Visual.ly titled 45 Ways to Communicate Two QuantitiesThe class split into teams of 3 people each and solved the problem in various ways, all of which can be seen on this Flickr page. My group solved the problem by deciding that the numbers 10, 30 and 60 represented specific colors on the HSB spectrum. At 100% saturation and 100% brightness the hues represented red(10), orange(30) and yellow(60). We were lucky that some of the materials matched these colors. For the second dimension we attempted to work proportionally as 10+30+60=100. So the proportions are 1/10, 3/10 and 3/5. We used a pipe cleaner 30mm long and divided it into sections according to these proportions which were 3mm, 9mm and 18mm. We then were able to use this as a guide for the amount of each colored material as is evident in the photo above. In the photo above red takes up about 10% while orange takes up about 30% and yellow takes up about 60% of the total materials used. Given more time than 20 minutes this data-viz could be refined and made more accurate. There are an infinite number of iterations to visualize such a concept as is evident in my other classmate’s work on the Flickr page.