Wednesday, May 30, 2012

Sent! Fast Company's Innovation by Design Awards

Alright, I've just now packed and shipped IDV's submission to Fast Company's Innovation by Design Awards competition.  I submitted the United Zipcodes of Craigslist, which was FastCo's infographic of the day back in March.  It was a zip-code agglomeration of the United States of Craigslist chalkboard map.



Here's the entry description...

The United Zipcodes of Craigslist came about as a follow-up resource for a preceding infographic, the United States of Craigslist.  The project came about as a means of identifying practical economic zones irrespective of political boundaries.  One clear source of persistent geographically localized economies is Craigslist.  The “sites” defining the ecosystem and dispersion of the Craigslist online community are the location of a great deal of pragmatic peer-to-peer commerce, but are themselves only a named regional entity.  How might these effective trade zones crisscross the United States?  If I were to drop out of the sky and brush myself off, in which Craigslist tribe’s turf would I find myself?  This was addressed in the initial “chalkboard” map visualization, the United States of Craigslist, within which the Voronoi method was used to grow purely digital boundaries around Craigslist “site” seeds.  This map was released publicly and soon questions started coming in around more immediately useful formats of the results –most commonly requests for a list of US Postal Service zip-codes corresponding to each region.  This was an opportunity to create the follow-up visualization, the United Zipcodes of Craigslist, which was featured as Fast Company’s March 5, 2012, Infographic of the Day (http://www.fastcodesign.com/1669120/america-charted-according-to-craigslist).  This map assembles these socio-economic zones from the pile of zip-code area bricks (where zip-codes, and therefore residents, exist).
What sorts of things can this, and other methods of constructing pragmatic economic zones, tell us?  It gives us a regional framework that, while ethereal, is in many ways more meaningful than the sometimes arbitrary delineations of political boundaries.  Over time these socially-constructed regions, and others like them, can and will shift, providing a fresh and scalable means of seeing and understanding communities.


Thursday, May 24, 2012

Who Cares About the Facebook IPO? Let's Ask Twitter...


IDV uses Twitter's throttled stream over time to get a general sense of overall global tweeting density, which we then dice up into boxes of roughly equal tweet volume (the Fischer method).  Using these equi-tweet boxes as a polygon baseline normalizes for the really uneven distribution of humans (and therefore tweets) over the globe.  Then, a thematic shading of these boxes based on the prevalence of any keyword within can then give us a visual sense of the relative proportion of a phenomena, geographically distributed.  In this case we used social media data to gauge the buzz about other social media.  We took a look at the proportion of tweets that included "facebook" or "fb" on the day of the Facebook IPO and painted the boxes accordingly.  We were surprised by some of the areas and not surprised by others.  What do you think?

Wednesday, May 23, 2012

Tornado Tracks



Got this data from NOAA via the spectacular Data.gov.  It tracks 56 years of tornado paths along with a host of attribute information.  Here, the tracks are categorized by their F-Scale (which isn't the latest and greatest means but good enough for a hack like me), where brighter strokes represent more violent storms.

Animations and Stuff
Check out the interactive version here.
Check this tornado tracks map broken out by F-Scale here.
Check this tornado tracks map broken out by seasonality here.
Check this tornado tracks as a print, available here.
Animated by month or by year.
Here is a more recent breakout of tornadoes by the direction they traveled.


Update
Here is an updated version of the Tornado Tracks map with newer data, including the horrible 2011 season.  Thanks to NOAA's http://www.spc.noaa.gov/wcm/ for the historical csv coordinate dataset and Adam Hill for development.  Interestingly ('excitedly' if you are a supernerd), while the original was a manual creation, this update is based on a raw export from Visual Fusion.  Export your own from the interactive version, here.

Click image for waaay higher resolution options, or click here to make your own map in the interactive version.

Other Sort-Of Related Maps
If this map is interesting to you, you might be amenable to plunging down the rabbit hole of related visualizations...
























Freaky Friday 2012.0 Projects


As promised, here is a summary rundown of projects that IDVers conjured for Freaky Friday 2012.0.  Lots of fun, lots of promise, and plenty of ideas for version 2012.1!  Here are the projects in the order that they were presented...

Merav
Merav created a training tutorial (sample module along with documentation) that walks a user through creating a data-driven filter control.  The filter presents the user with choices that are populated dynamically from the data source.


Nahim
Nahim created a data-view maker that allows the user to aggregate quantitative data of a feed by choosing an attribute and an aggregate function.  He also created a means to filter by attribute values in a non-database feed.


Adam F.
Adam created a filter-building engine that scans the attributes and values of a feed and presents a set of client-side data filters.  With an eye toward UX, filters are automatically shown as lists, drop-downs, and sliders.  He also threw in pie and sunburst charts.


Abhinav
Abhinav created a Pseudo 3D Basemap Overlay method that stacks multi-floor image overlays into a vertical tower that honors map tilt/rotate.


Abhishek
Abhishek created the GeoChi, a method of calculating the weighted geographic centroid on-the-fly given a constellation of point inputs with weight factors added by the user in the client.


Leo
Leo wrote a VF connector to the document-oriented NoSQL database system, MongoDB, which is used by many large enterprises.  The connector has read/write capabilities so MongoDB records can be created/managed/deleted directly from the VF client.


Marketing Team
George, Deb, and Chris F. collaborated on a proposal to increase the confidence and clarity of alerting by leveraging more spatially-precise weather warning areas to reduce false positive weather alerting and  mocked up a rules-based method of alert prioritization so that top-tier events are given visual and auditory prominence.


Chris A.
Chris A. used standard SharePoint capabilities and a little magic to replicate the more flexible relational data structure of a database by using workflows and merging values in a SharePoint feed’s available attributes and styling rules.  Additionally, he intercepted the feed’s image preview provided by VF for SharePoint and inserted it into the details panel for any item belonging to that feed.


Justin
Justin improved the Basemap Tiling Scheme to efficiently pre-load batches of imagery above a user’s position to smooth out and reduce basemap gaps during rapid map navigation.


Adam H.
Adam H. created the Super-Ultra-Mega-Find, a one-stop text input that shows feeds, feed items, fly-to options, and ad-hoc feed results as you type.  The results are displayed in a federated list in the details window.


Daniel
Daniel created Remix, an automated feed and attribute detection and configuration engine that presents a sequential UI for discovering data sources and attributes, allowing the user to define visualization preferences in VF.


Neeta
Neeta created a Dispatch Soundscape that pulls in live streaming emergency response audio of public dispatch bands by mousing over US Counties.  Police bands are given play-priority on mouse-over and other bands are provided on-click in the counties’ details panel.


Andy
Andy created Backgrounder, a VF Desktop Image Cycle that converts the live status of any VF/VCC app into a series of images that feed a Windows desktop image slideshow background.


Tuesday, May 22, 2012

Freaky Friday 2012.0 Results!


It was a rip roaring show and IDVers put together a pile of fun and inventive stuff.  After a rigorous and highly formal vote of peers, here are your Freaky Friday 2012.0 winners…



Most Original
This cool and fascinatingly eccentric topic came right out of nowhere and hit stunned voters squarely in the nose.  A tie (ironically?)!


Neeta made a Dispatch Soundscape that pulled in live streaming emergency response audio by mousing over US Counties.


Andy made a VF Desktop Image Cycle that converts the live status of any VF/VCC app into a continuous desktop background slideshow.

Most Needed
Upon the unveiling of this project, an audible steam hiss will be heard as the idea comes into contact with red-hot demand, followed quickly with a great sigh of relief and the clapping of high fives.  This project scratches a persistent itch.


Justin improved the Basemap Tiling Scheme to efficiently pre-load batches of imagery above and below a user’s position to reduce basemap gaps during navigation.

Most Marketable
This project is the most likely to be seized upon by eager peers of all sorts, poured enthusiastically into a messaging crop duster, and blasted into a parched market with great relish.


Abhinav created a Pseudo 3D Basemap Overlay that stacks multi-floor building floorplans into a vertical tower when the map is tilted or rotated.

Most Promising
Maybe it isn’t in a state of demonstrable fruition yet, but this project lives within that honorable intersection of ‘beautiful in concept, clear in intent, and feasible in scope.’


Daniel created Remix, an automated feed and attribute detection and configuration engine that presents a sequential UI for discovering and defining visualization preferences in VF.

Best of Show
Of all the projects, this stands out as generally the most awe-inspiring.  Maybe it was one of the categorical winners, or maybe it wasn’t somehow, but in any case this is the project that gets hoisted atop the shoulders of rapturous teammates and paraded around the field.  Then, a couple decades go by and ‘oh, I was Best of Show this, and Best of Show that’ and everybody has heard the tired old stories of the glory days and wishes the Best of Show topic would just give it a rest already.


Adam H. created the SuperUltraMegaFind, a one-stop text input that shows feeds, feed items, fly-to options, and ad-hoc feed results as you type.

Each category champion was the recipient of a lavish gift bag containing...

  • A securely stapled and heavily graffitied dollar bill,
  • a tube of the Glue of Resourcefulness,
  • a Spritzer of Skill,
  • a Glow Stick of Insight,
  • a Hammer of Persistence,
  • and Unrepentant Bragging Rights.
  • The Best of Show winner received the Sweatshirt of Triumph, upon which are the signatures of the vanquished.


Thanks to all who participated and supported our first smashing Freaky Friday!  I’ll post a more detailed visual recap of all the projects soon.  In the meantime, get ready to live.

Thursday, May 17, 2012

Mental States

Here's a bucket map of tweets by proportion of bored tweeters, or excited.  As you can see there are pretty distinct geographic trends around this relative state of mind.


Why:
Social networking data can provide lots of information, and fast.  Plotting the prevalence of a keyword from among the population of tweets is an interesting way to normalize for the old problem of first order trends in a dataset.  The raw tweet point locations of bored and excited would just look like a sampled distribution of tweets in general, by mapping proportion within equally populous zones you get a sense of actual variability.  It's also interesting to choose a set of related keywords so you get a comparative graphic of related (in this case, opposite) cultural phenomena.  In this case it turned out to be the relative enthusiasm of the US in a random day in the spring.

How:
Each zone represents an area of roughly 10,000 geo-located tweets over the course of one day (May 11 or thereabouts, I think).  This method was the idea of Eric Fischer, who mapped global subdivisions of equal tweet frequency.  We set the number of tweets per zone to 10,000.  Aside: in time, I'd like to play with the idea of an algorithm that stepped through increasingly higher tweet numbers to see if the stepwise process of large to small buckets might make more spatially compact zones (zones of super-dense content tend to be understandably narrow along one dimension but really long in the other).  Daniel Briggs here at IDV used Eric's script to generate the boxes using one day of data from Twitter's sweet sweet API.
Using this underlying map as a denominator representing the general tweet population, we picked out tweets containing "bored" or "excited" and thematically mapped their proportion in Visual Fusion.

We're really interested in digging into the spatial power of social media to find structure in events as they happen.  This shows the utility of aggregation, but here's another example where physical movement is illustrated with discrete tweets as seen in the context of geography and time.


Wednesday, May 16, 2012

Freaky Friday


In a fast-paced office environment, it can be easy to get sucked into the secure trap of project complaisance.  There is always a stack of tasks with your name on them and you never really have the time to clear them all out.  Along the way you stub in mental post-it notes for awesome ideas or better ways of doing things in hindsight, and they fade over time.

But remember how awesome it is when every once in a great while your inbox is empty and you are left relatively to your own devices?  That's when the good stuff happens!  All those thoughts for inventions, alternatives, and improvements that you've had start popping up on your shoulder dressed like little angels or devils, but either way it is a stolen moment to experiment and invent!  If you feel this way, chances are good it's a pretty big reason you got hired in the first place -and it's a good idea for your employer to let you free range once in a while.  The surest way to go stale as a business is to keep your staff busy.

So this Friday IDV Solutions has cleared out the calendar and is hosting a no-holds-barred make-fest for employees.  And it is going to rule.  Here are the categories chosen by the staff, by which they will judge each other the following Monday...Reckoning Monday.  Winners will receive cheap junk and unrepentant bragging rights.  We'll post updates as the day goes by, so get ready to live and stay tuned at @johnnelsonidv and @idvsolutions...

And the categories are...


Most Original
This cool and fascinatingly eccentric project came right out of nowhere and hit stunned voters squarely in the nose.

Most Needed
Upon the unveiling of this project, an audible steam hiss will be heard as the idea comes into contact with red-hot demand, followed quickly with a great sigh of relief and the clapping of high fives.  This project scratches a persistent itch.

Most Marketable
This project is the most likely to be seized upon by eager peers of all sorts, poured enthusiastically into a messaging crop duster, and blasted into a parched market with great relish.

Most Promising
Maybe it isn’t in a state of demonstrable fruition yet, but this project lives within that honorable intersection of ‘beautiful in concept, clear in intent, and feasible in scope.’

Best of Show
Of all the projects, this stands out as generally the most awe-inspiring.  Maybe it was one of the categorical winners, or maybe it wasn’t somehow, but in any case this is the project that gets hoisted atop the shoulders of rapturous teammates and paraded around the field.  Then, a couple decades go by and ‘oh, I was Best of Show this, and Best of Show that’ and everybody has heard enough about all the tired old stories of the glory days and wishes Best of Show would just give it a rest already.

Thursday, May 3, 2012

A Twitter Anatomy of a Protest

Here's a visualization of mid and lower Manhattan on MayDay, 2012, plotting the when and where of tweets containing the keywords, MayDay and Occupy (representing a healthy mix of supporters, detractors, and everybody in-between).  The visual coordination of three dimensions of data: location, time, and topic, provides an up-to-the-second profile of a social event as it forms, moves, and dissipates.


Social media, in this case a sampling from Twitter, can provide excellent insight into the anatomy of an event as it trends in space and time.  In addition to the ability to analyze an event after it has happened, and of course the general benefit of seeing a movement as it is happening, sometimes the combination of these visual dimensions may allow a viewer to reasonably extrapolate or predict some organic movements of  social phenomena before they happen.

This visualization (and the series of insets below) shows the Bryant Park staging area as tweets converge around the meeting point, displays the tight linear trend of the march's progress south to the business district, and finally the dissipation cloud around the NYSE that evening.  The time profile, meanwhile, shows the buildup of tweets using these keywords, plateauing at the time of overall movement, peaking at the end of the workday, then smoothly dropping off in frequency.

A gathering of topically related (#MayDay, #Occupy) tweets cluster around the Bryant Park, the physical meeting point of the march.  The timeline shows the number of tweets growing with activity.

These event tweets show a strong linear path, relating to the march proceeding south along Broadway.  Tweet volume plateaus.

Finally, the related tweets coalesce around Wall Street, the final destination of the protesters, where the tweet volume peaks then gradually diminishes.



 

Important Temporal Visualization

Wrap your brain around this, my friends!!