allourideas
Feb 08
first paper
We’ve just posted our first paper about the research behind allourideas.org. Comments and suggestions are welcome.
Wiki surveys: Open and quantifiable social data collection
Matthew J. Salganik and Karen E. C. Levy
Abstract: Research about attitudes and opinions is central to social science and relies on two common methodological approaches: surveys and interviews. While surveys enable the quantification of large amounts of information quickly and at a reasonable cost, they are routinely criticized for being “top-down” and rigid. In contrast, interviews allow unanticipated information to “bubble up” directly from respondents, but are slow, expensive, and difficult to quantify. Advances in computing technology now enable a hybrid approach that combines the quantifiability of a survey and the openness of an interview; we call this new class of data collection tools wiki surveys. Drawing on principles underlying successful information aggregation projects, such as Wikipedia, we propose three general criteria that wiki surveys should satisfy: they should be greedy, collaborative, and adaptive. We then present results from www.allourideas.org, a free and open-source website we created that enables groups all over the world to deploy wiki surveys. To date, about 1,500 wiki surveys have been created, and they have collected over 60,000 ideas and 2.5 million votes. We describe the methodological challenges involved in collecting and analyzing this type of data and present case studies of wiki surveys created by the New York City Mayor’s Office and the Organisation for Economic Co-operation and Development (OECD). We conclude with a discussion of limitations, many of which may be overcome with additional research.
Jan 20
Wikipedia Banner Challenge: Results
Congratulations to Wikipedia for a successful fundraiser. They raised 20 million dollars with donations from more than one million people. Now that the fundraiser is complete, we have archived the Wikipedia Banner Challenge; you can still vote and upload new banners, but those contributions will not be recorded. Below, I’ll present some analysis of the data and provide links to the raw data so that you can analyze it too.
Read More
Dec 20
Wikipedia Banner Challenge
As you can tell from the banners appearing all over Wikipedia, their fundraiser is in full swing. Despite Wikipedia’s importance as a global resource, only about one-in-a-thousand Wikipedia readers donate. One way to improve that would be better banners, and that’s why we are launching the Wikipedia Banner Challenge, a website to collect and prioritize banner ideas for Wikipedia. You can participate by voting on banners and suggesting new ones. It is quick, easy, and even a little fun.
The Wikipedia Banner Challenge builds on previous innovative efforts by Wikipedia to involve their community in the design of the fundraiser, especially during the 2010 fundraiser. In a continuation of that community-driven spirit, Wikipedia announced on their blog that they will be watching the results from the Wikipedia Banner Challenge closely and will use some of the most promising banners during the fundraiser. In other words, your banner could appear in front of Wikipedia users around the world.
The Wikipedia Banner Challenge is a customization of the core allourideas.org code that was completed in about a week by two awesome web developers: Chap Ambrose and Luke Baker. In addition to furthering our research and helping Wikipedia, we hope that this project will also encourage others to customize our open source software for their own purposes.
Wikipedia has always been an inspiration for this research, so we are very excited that they decided to use allourideas for such an important project. Wikipedias, if you want better banners, give us your ideas.
Here are links to more information about:
* The 2011 fundraiser
* The 2010 fundraiser
* Banner ideas from 2010 and banner test results
* Banner ideas from 2009
Dec 14
Governor Genro tops President Obama on Citizen Feedback: “The Governer Asks” vs. “Open for Questions”
Something neat is happening in Porto Alegre, Brazil today. Governor Tarso Genro, of the state of Rio Grande do Sul, is meeting with some of his constituents. Of course, that’s pretty normal; governors meet with constituents all the time. What is neat is how those constituents were selected. They are not the ones with the most money or influence, they are the ones with the best ideas.
These 50 constituents were selected to meet with Governor Genro through a process called Governador Pergunta (The Governor Asks). The process started when citizens suggested 1,300 ideas related to five different aspects of health care (e.g., access to care, family health). Next, the Governor’s office launched a major public outreach campaign to encourage residents to prioritize these ideas through an online voting process. To broaden participation, there were public events and even a “voting van” packed with Internet-connected computers that drove around the state. In just 30 days, Governador Pergunta collected 120,000 votes, and these votes were used to select the top 10 ideas in each of the five categories.
To readers in the US, Governador Pergunta might sound like President Obama’s Open for Questions, and the two did have the same admirable goal: to increase public participation in government. But, there were important differences in their implementation that lead me to conclude that Governor Genro’s Governador Pergunta topped President Obama’s Open for Questions.
The first big difference between the two projects was their voting mechanisms. Here’s what they looked like:
Governador Pergunta
Open for Questions
Open for Questions used single-column, approval voting. Visitors to the site could find the items that they wanted and then vote for them. Governador Pergunta used pairwise comparison, meaning that voters were presented with two options and are asked to choose between them. These mechanism may seem similar, but the Governador Pergunta voting system is better than Open for Questions in important ways. (Disclaimer: Of course, readers of this blog know that pairwise comparison is the voting mechanism for allourideas, and it turns out that that Governador Pergunta used the allourideas pairwise API. But, more on that later.)
One reason that the voting mechanism in Governador Pergunta is better is that voters made their decisions independently; they had no information about how others had voted. In Open for Questions, in contrast, voters made their decisions interdependently; items were sorted by popularity and this popularity was shown to voters (see screenshot above). This type of interdependent voting system, unfortunately, can lead to strong and unpredictable fads where some ideas get additional support mainly because they had been supported in the past. As I’ve shown in some earlier web-based experiments, the stronger the interdependence of decision-making, the weaker the relationship between underlying quality and ultimate success. In other words, interdependent voting systems are not good for finding the best ideas.
Further, the pairwise comparison voting mechanism used by Governador Pergunta is more manipulation resistant. Recall that in the approval voting system used in Open for Questions, the voters chose which items to consider. This feature makes it easy for a small group of people to rush to a single idea and push it to the top. This weakness was quickly discovered and exploited by the National Organization for the Reform of Marjuana Laws (NORML). In the midst of a financial crisis and national debate about health reform, many of the highest scoring items in Open for Questions were focused on the legalization of marijuana.
With a pairwise comparison voting mechanism, such as the one used by Governador Pergunta, it would have been much harder (but not impossible) for NORML, or any other group, to skew the results because a voter would have had to cast many, many votes before she would get a chance to vote for the idea she wanted to push to the top. Whatever you think about the fairness of marijuana laws in the US, having a system of public participation that is open to manipulation by a small group is clearly not ideal.
Finally, in addition to using a superior voting system, Governador Pergunta topped Open for Questions in another way: it was open-source. Just as black-box electronic voting machines threaten public confidence in elections, so to do black-box systems for other forms of public participation in democratic governance. Any effort to make government more open and transparent using processes that are not open and not transparent seems destined to fail. The source code for Governador Pergunta and the source code for the Pairwise API, used by used Governador Pergunta, are both open-source. The Governor’s team and I hope that other public officials will build on our work to develop even better ways of making government more open, transparent, and effective.
Note: I posted a very similar version of this article on the Freedom to Tinker blog.
Dec 07
great news: another grant from Google
Our project has just been awarded another Google Faculty Research Award. This grant will allow us to continue our research and keep improving allourideas.org. Thank you Google.
Nov 23
new code patch from the community
Thank you to Lucas D’Avila from Brazil for submitting an improvement to our code-base. Here are the details of the patch.
All Our Ideas is an open source project, and we welcome contributions from everyone in our community. [The image of Tux the Linux penguin is from Wikimedia Commons.]
Nov 02
server improvements
We took the site down today for about an hour for some server improvements. Everything went as planned, and we are now running Passenger 3. This should lead to improved performance, especially when we have multiple concurrent sessions.
Thank you to the good folks at Phusion who released Passenger 3 open-source.
Oct 19
occupy wall street
Members of the Occupy Wall Street movement have started using allourideas.org to help articulate goals. You can:
Oct 10
The Junior League and all our ideas
The Association of Junior Leagues International Inc. is a non-profit that improves communities through volunteer projects and builds the leadership skills of its 155,000 members. To provide additional training opportunities to members, the Junior League recently started a Wednesday Webinar series, and the first challenge for Becki Fleischer, the leader of the series, was to decide on speakers. Of course Becki could have sat down at her desk and created a list of people. But, these would have been the people that she thought were best, and what Becki really wanted was a list of the people that her members thought were best.
Becki had heard about allourideas.org from a Beth Kanter blog post about crowdsourcing, and she decided to give it a try. 24,000 votes and 60 ideas later, the choices were clear. Shar McBee, the top scoring suggestion, is already booked to give the webinar on April 18th, and Becki is in the process of inviting other members from the list.
Sep 11
allourideas in hebrew
I am happy to announce that the voter-facing portions of the site have now been translated into Hebrew. Thank you to the volunteer translator Uri Shwed.
All Our Ideas is now available in seven languages other than English thanks to the great work of volunteers. If you would like to help translate the site into another language, please send me an email.
