This paper will give an introduction into the rise of crowdsourcing, its methods and the controversies surrounding it.
Class Action Against Crowdsourcing
Something is brewing in the world of digital labour. In October 2012, online worker Christopher Otey filed a class action lawsuit against the US based company CrowdFlower, one of the largest crowdsourcing platforms for the completion of so called ‘micro-tasks’. CrowdFlower has a reserve army of, as they claim, millions of workers that can be hired instantly to process data. According to its CEO Lukas Biewald the company can hire 10.000 people in an hour and they do hire up to 3 man-years of work on a daily basis (Biewald). Through the pending lawsuit, Christopher Otey is challenging the companies failure to pay the minimum wage under the Fair Labor Standards Act to its US workforce and his lawyers are now searching the web for other underpaid members of the online crowd who might want to join the class action. CrowdFlower’s lawyers point out, however, that Christopher Otey did his work completely voluntarily and that he and all the other ‘cloud-workers’ are not employees but free contractors. The case is still open, but it has the potential to shake the foundations of a business model that has been mushrooming around the globe over the last five years. Crowdsourcing has become a huge industry, with the size of the workforce doubling each year and revenues rising by 75 percent per annum (Massolution 9-19).
Contrary to a commonly held view of crowdsourcing as a transfer of low-skill work to low cost locations, our analysis shows that more than half of all the crowdsourcing workers live in North America and Europe and workers are generally very well educated. Almost half have a bachelor degree and only 5% are truly low skills workers with only an elementary education. (Massolution 19)
The website Crowdsourcing.org which issues the report and understands itself as a hub for the industry already lists over 2000 different websites for crowdsourcing and crowdfunding. (Crowdfunding is an important subcategory of crowdsourcing in which the crowd simply is the source for money, but in this paper I will focus on the crowd as a source for data, knowledge, ideas and, most importantly, work.) To some, crowdsourcing is a neutral umbrella term that describes new processes of distributing labour; to others it is the exploitation of cheap or free labour with detrimental effects for workers and professions. The questions are: Is crowdsourcing exploitative even when all participants are volunteers and know the conditions? Is it labour when people do the work as a hobby? Is crowdsourcing inherently unethical or is it just a question of how the parameters are configured? And how can national labour laws tackle a global phenomenon? It is not easy to evaluate crowdsourcing because of its varying definitions and methods. The deal between those who do the work and those who profit from it varies from platform to platform. The different approaches in crowdsourcing are scattered across a spectrum that reaches from productive leisure and play over altruistic volunteering to precarious labour. In the following, I will outline some of the aspects that are relevant in regard to the ethical dimension of crowdsourcing.
From the Empowerment of the User to the Harnessing of the Crowd
Buy it, use it, break it, fix it,
trash it, change it, mail – upgrade it,
charge it, point it, zoom it, press it,
snap it, work it, quick – erase it,
write it, cut it, paste it, save it,
load it, check it, quick – rewrite it,
plug it, play it, burn it, rip it,
drag and drop it, zip – unzip it,
lock it, fill it, call it, find it,
view it, code it, jam – unlock it,
surf it, scroll it, pause it, click it,
cross it, crack it, switch – update it,
name it, rate it, tune it, print it,
scan it, send it, fax – rename it,
touch it, bring it, pay it, watch it,
turn it, leave it, stop – format it.
(Daft Punk, Technologic, 2oo5)
Every day, we click our way through an endless succession of micro-tasks. Isolated, they are only meaningful to us, but in aggregated form, they are of great value for companies. On the lowest level, these tasks are almost under the threshold of perception: Surf it, scroll it, pause it, click it. But all this creates data for Google & Co to further develop their algorithms and sell personalised adds. Write it, cut it, paste it, save it. Other tasks already demand more engagement, they not only create data but content be it for self-expression or as a service to others. Amateurs online write articles for Wikipedia, moderate help forums, debug open source software and make valuable contribution to sciences such as astronomy to ornithology. With increasing complexity, these task stop being micro and demand a high level of engagement and expertise. They eventually become indistinguishable from work. The lines between amateur and professional, between consumption and production, usage and creation, play and labour, have been continuously blurred in post-industrial production and especially online. Portmanteaus such as ‘prosuming’ and ‘produsage’, ‘playbour’ and ‘weisure’ or the ‘pro-am revolution’ have tried to express this. It is the collapse of the boundaries between the two domains, that makes the valuation of appropriate remuneration is so tricky. It is a spectrum, with hybrids that are predominantly fun on one side and hybrids that are predominantly work on the other. The most drastic illustration for this strange amalgam is probably ‘goldfarming’, where the fun of playing a game is perverted into pointless virtual drudgery for real world currency. While the criticism used to be about the low quality of amateur work (Keen), the controversy has shifted. Since it is evident that these free contributions can be of great value also economically, the question is now who owns them and is therefore entitled to make a profit of them?
Let’s take the harvesting of data: It is well known that the services of Google, Facebook and the like are not actually free, payed for with personal data. In other words, if the service is free, the users are the product being sold to advertisers. With the accusation of exploitation already looming in the background, Nicholas Carr has described Facebook’s business model as ‘digital sharecropping’ (Carr). Carr refers to what Facebook calls ARPU or average revenue per user, which was at $5,11 for 2011. Not much for a single user, but they got over a billion of them. It is puzzling that the majority of users obviously prefers sell out their privacy instead of paying a relatively small fee for the maintenance of the social network, but privacy issues aside, I wouldn’t call Facebook’s business-model exploitative. The value creation of the users happens as a side-effect of their activities and in return they get a service that they use intensely.
User-generated content became the central idea of the so called Web 2.0, a term popularised in 2004 by publisher Tim O’Reilly. The new version of the internet, so it was said, had become more collaborative and participatory. Tim Berners-Lee strongly objected the whole notion of a Web 2.0 — getting people together to collaborate online was exactly what he developed the world wide web for in the first place, and critics such as Trebor Scholz have shown, that the proclaimed novelty of the Web 2.0 was deceptive. It was actually just a clever marketing label from which even O’Reilly distanced himself eventually (Scholz, “Market Ideology and the Myths of Web 2.0”). But still, something had changed on the internet around the time that Web 2.0 rose to fame. The masses had arrived online and with them came a revival of the notion of the crowd. What’s more, after the burst of the dotcom bubble in 2000, the enthusiasm for e-commerce had cooled down for a few years. What united many of the websites that arose from the ashes was that they all found ways to let the newly arrived masses of users produce the content for each other. The companies only had to provide the infrastructure, the tools, the stage and Web 2.0 delivered the narrative for this transformation.
In the case of Amazon, users already contributed ratings, reviews and recommendations but they did not influence the actual products. With the launch of Second Life in 2003 and, most importantly, YouTube in 2005 the concept of user-generated content was elevated to a new level. Now, the users also created the core product. Wikipedia had started in 2001, but it was between 2004 and 2006 that it was growing exponentially. All this contributed to a great hype about the empowerment of the user, which peaked in December 2006 when Time magazine made You the Person of the Year, showing on the cover a mirror foil, framed by a YouTube player. Below it read: “Yes, you. You control the Information Age. Welcome to your world.” In the corresponding article, Time continued: “this is not the Web that Tim Berners-Lee hacked together […] and not even the overhyped dotcom Web of the late 1990s. The new Web is a very different thing. It’s a tool for bringing together the small contributions of millions of people and making them matter. […] It’s about the many wresting power from the few and helping one another for nothing” (Grossman). It turns out that this was, at least to some extent, an illusion. While the many do indeed help each other for nothing, the power today seems to be back firmly in the hands of the few. Users had much more control over their data and content before everything moved over from the personal homepage to the servers of the global aggregators and social networks.
The rise of crowdsourcing
Welcome to the age of the crowd, (where) […] distributed labor networks are using the Internet to exploit the spare processing power of millions of human brains. […] The labor isn’t always free, but it costs a lot less than paying traditional employees. It’s not outsourcing; it’s crowdsourcing. (Howe)
In 2006, in an article for Wired, the journalist Jeff Howe combined the terms crowd and outsourcing to describe the new form of cheap labour online. The framing of labour as spare processing powers suggests that something is put to good use that would otherwise be wasted. At the same time, this already has a dehumanising tone to it and is clearly not about the empowerment of the individual anymore. Now that plummeting costs and widespread dissemination of technology had made the aggregation of human activities and mental capacities feasible on a massive scale and people started thinking about how to put this new resource to good use. The attempt to fathom the dimension of the yet to exploit resource quickly led to astronomic calculations. Internet guru Clay Shirky wrote in his book Cognitive Surplus: how technology makes consumers into collaborators that “the world’s educated population has three trillion hours of free time each year” (Shirky, Cognitive surplus 27). His vision is to use at least a fraction of the time otherwise wasted in front of the TV for more productive causes. A popular example in these calculation is, that it took ‘only’ about 100 million man-hours to create the Wikipedia.
Along these lines, the game-designer and author Jane McGonigal has pointed to the total number of hours people played World of Warcraft, which in 2011 accumulated to 5.93 million years, just for this one game alone. According to McGonigal, people also played 3.5 million years of Bejeweled and 250.000 years of Halo and so on. Her vision is to create games that have a positive influence on real world problems such as health and sustainability and she also created games with that ambition (McGonigal). The problem with this approach is, that it cuts both ways and the so called ‘gamification’ of work, that is to say, the introduction of points and badges and other virtual reimbursements has become a popular tool in crowdsourcing to ‘pay’ the contributors without having to pay cash. Gamification propels competition and ambition among the workers and transform the feeling of loss of time into a feeling of achievement and progress (Herz). Because of its manipulative power in the business context, media philosopher Ian Bogost has suggested to better speak of this approach as ‘exploitationware’ (Bogost).
Luis von Ahn, researcher at Carnegie Mellon university has developed a very special from of harnessing the ‘cognitive surplus’ or ‘human computation’ as he calls it. He is the inventor of CAPTCHA, those distorted letters that we have to type in when we create a new account on a website. With that, we prove to the computer that we are human and allow the machine to keep spambots out of the system. Von Ahn had observed, however, that spam companies would hire humans to type in these squiggly letters as a full time job. People would get paid $2.50 per hour and in that time would solve about 720 CAPTCHAs. Access to a new account from which to send spam would cost the company only a third of a cent. This lead von Ahn to the invention of reCAPTCHA, now commonly used across the web. In this new system, we help with the digitisation of books, whenever we type in the letters, we recognise a fragment of a scan that the computer could not decipher. As von Ahn says that he wants to “solve large-scale computational problems and/or collect training data to teach computers many of these human talents. To this end, I treat human brains as processors in a distributed system, each performing a small part of a massive computation.” The interesting thing about Ahn’s method of crowdsourcing is, that people often don’t even know that they are accomplishing a useful task when they solve the reCAPTCHA.
All these examples for the aggregation of usage data, unconsciously performed micro-tasks, and user-generated content are examples for crowdsourcing in a broader sense. In a narrow sense, “crowdsourcing represents the act of a company or institution taking a function once performed by employees and outsourcing it to an undefined (and generally large) network of people in the form of an open call” (Jeff Howe 2006 on his website). This is not about skimming off something that is already there, this is about replacing proper jobs with often precarious labour. While business gurus love it and even the most derivative crowdsourcing websites repeatedly get awards and accolades for their innovative ways to slash costs of production, many workers in the creative industries feel threatened by crowdsourcing. It is not only those whose jobs are in danger that see crowdsourcing critically. In an interview with the San Francisco Chronicle, Jimmy Wales said:
One of my rants is against the term “crowdsourcing,” which I think is a vile, vile way of looking at that world. This idea that a good business model is to get the public to do your work for free — that’s just crazy. It disrespects the people. It’s like you’re trying to trick them into doing work for free. (Wales)
This might come as a surprise from the founder of Wikipedia, after all, the online encyclopedia wouldn’t exist without people constantly contributing their work for free. But the essential difference is that as with open source software, the volunteer work done by the community creates a free resource for the commons. It becomes useful to even more not less people. In the case of most crowdsourcing projects, the participants have no direct use for their contributions and usually give away their intellectual property rights automatically, therefore the open process end with a closure. The final owner is someone outside of the community and the re-introduction of the term crowd as a substitute for the term community is already very telling in this regard. A crowd is other people.
The Reinvention of the Crowd
Until the end of the twentieth century the term crowd had a clear connotation. It was a disorganised, unruly gathering of people with a dynamic that could quickly turn a group of cheering spectators into a raging mob. This image of the crowd gained particular relevance during the Industrial Revolution when a steep rise in population combined with massive urbanisation led to overcrowded tenements, people densely packed under grim conditions, never far away from taking it to the streets. The term was also used its more abstract sense by Charles Mackay, who published Extraordinary Popular Delusions and the Madness of Crowds in 1841. The book was mainly a detailed description of historic hypes and speculative bubbles such as the tulipomania — cases in which groupthink and information cascades had caused whole populations to collectively (and metaphorically) run into the wrong direction. This crowd didn’t have to be in one physical place. But the predominant meaning of the crowd was that of a group of people defined through proximity, acting as one and potentially dangerous to those in power. The crowds became a field of study and in 1895, Gustave Le Bon published The Crowd: A Study of the Popular Mind. Having studied the aftermath of the French Revolution, the sociologist was convinced that by joining a crowd, every human would degenerate and succumb his will to the brutish and animal-like hive mind. The sum of people would always be less then its parts — a crowd might occasionally do some heroic act, but it could never act in any intelligent, trustworthy or productive way. “Civilisations as yet have only been created and directed by a small intellectual aristocracy, never by crowds. Crowds are only powerful for destruction” (Le Bon 10). With the spectre of democracy haunting Europe, the question for Le Bon was how to keep the crowd at bay and influence it in favour of those in power. Part of his study therefore reads like a manual for crowd manipulation. Stephen Reicher, a modern day expert on crowd psychology writes about Le Bon:
Certainly, Le Bon influenced a plethora of dictators and demagogues, most notoriously, Goebbels, Hitler and Mussolini. This influence was not in spite of but rather an expression of Le Bon’s intentions. He repeatedly urged contemporary establishment figures to employ his principles in order to use the power of crowd for, rather than against, the state. His perspective matched the concerns of the age in their entirety: fear and fascination in equal measure; denigration of the collective intellect, harnessing of collective energy.
[…] The majority of his crowd text is, in fact, essentially a primer on how to take advantage of the crowd mentality, how to manipulate crowds and how to recruit their enthusiasms to ones own ends (Reicher 5-6).
Occupy Wall Street, the London Riots and the Arab Spring are recent examples, that the classic form of the crowd, the one that Le Bon studied and feared, still exists today, but since the turn of this century we have experienced a split in our image of the crowd. There are now two stereotypical crowds that almost look like a mirror images: the revolutionary and destructive crowd in the streets, fighting against oppression, and the docile crowd online, productively clicking in the hours. The challenge is not anymore how to suppress and manipulate its destructive power, but how to harness its collective intelligence.
The reinterpretation of the crowd was triggered by the journalist James Surowiecki who argued that “Gustave Le Bon had things exactly backward. If you put together a big enough and diverse enough group of people […], that groups decision will, over time, be ‘intellectually (superior) to the isolated individual,’ no matter how smart or well-informed he is.” (Surowiecki, introduction). His book The Wisdom of Crowds – How the Many are Smarter then the Few, published in 2004, not only echoes the early days of crowd psychology in its title, it also turned its core beliefs upside down and let to a paradigm shift in the predominant image of the crowd. Even though Jeff Howe did not mention Surowiecki in the original article on crowdsourcing he later acknowledged, that the book was an important influence for the coinage of crowdsourcing. Surowiecki was able to show with an array of research from various fields, that under certain conditions, a crowd would indeed deliver better results than any expert. This alone was a significant revelation, but it could only unfold its full potential in combination the other discovery already describes above, that on the internet, people where willing to do complex tasks for free. Wikipedia and Linux had become the undeniable proves of concept for what would have been unimaginable in theory.
Core Methods of Crowdsourcing: Micro-tasking vs. Contests
Crowdsourcing is sometimes used by companies on their own website as a one-off marketing stunt to engage costumers with a brand, for example by letting them design a new, temporary label. It also serves as market research tool to find out about opportunities for new products through involving the users. In its more elaborate form, crowdsourcing becomes a business in its own right. Specialised companies create online platforms, not unlike a beehive, to attract and accommodate permanent communities of workers and sell their produce. Their workforce is offered to external clients, and the platform owner makes a profit by taking a commission for all the work done by the hive. The methods for orchestrating the workforce of the hive vary greatly. Some owners put emphasis on the collaborative aspects inside the community and especially give incentives for cooperative behaviour, while others foster competition. Many platforms offer non-monetary gamification incentives, like virtual badges or credit points that give the contributors reputation in their community — others actually pay their workers. While the crowd is, by definition, not limited to a certain number, the money that is being paid out certainly is. Because everyone can participate it is practically not possible that everybody get paid in full. When money is involved, there are basically two different models. In the first model, the workers get micro-payment for repetitive micro-tasks, e.g. for categorising items or recognising something in an image a fraction of a cent is paid. In the second model, popular when the work is more complex and time consuming and can’t easily be split in tiny units, it is organised in the form of contests. In this model, many competitors do the same job at the same time but only one person gets paid in the end.
CrowdFlower.com, the company now faced with the class action law suit, is a typical example for the micro-payment model. The most prominent is Amazon’s Mechanical Turk, named after the historic chess robot that was actually operated by a human, hidden on the machine. Amazon describes its service also as ‘artificial artificial intelligence’. Essentially they tackle the same sort of problems as Luis von Ahn with his human computation. One of the most baffling applications in this area is ‘Soylent – a word processor with a crowd inside’. It is basically a plug in for MS Word which allows the user to assign parts of a text to the crowd of Mechanical Turkers for correction or shortening without even leaving the program (Bernstein). (The name, by the way, comes from the apocalyptic science fiction film Soylent Green (1973) and stands for a popular snack in that turns out to be made out of humans…)
The numbers vary from job to job and also depend a lot on their experience, but on average workers on micro-tasking platforms earn about $2 per hour, often less. This equals roughly the minimum wage of $2 in Beijing, and is a far cry from the US minimum wage of $7.25. The case Otey vs. CrowdFlower could therefore have a huge impact, depending how the judges decide. As Eric Mack of crowdsourcing.org points out, the case “challenges the assumption at the very foundation of crowdsourcing”, that “cloud workers” are not employees, it could undo or cement the whole industry in the US (Mack). When looking at the numerous debates online among those being crowdsourced, it is surprising to see that even though the workers do feel exploited, they also often defend the platforms and are worried that a change in legislation could take this last straw of income away from them. It is already difficult to imagine how to live from this form of labour in a developing country, but people actually try to do just that in the wealthiest nations. To give just one example of this, on August 25, 2012, a Jacqueline Parks comments on the website ConcurringOpinions.com under an article that discusses wether universities should continue to conduct research with participants through Amazons micro-task platform Mechanical Turk:
I am a Mechanical Turk worker. I am American, and our family has found ourselves in a hard place financially. I work at Turk for about 12 hours a day, and my average pay is $1.40 an hour. […] I keep working because we need money to be OK right now. I can’t wait until some unknown future date when I might find a better job. Matter of fact, we are at risk for having our electricity shut off, and I need $179 fast and am hoping to have that total deposited in my account soon. […] I do feel like a sweat shop employee. I do not make minimum wage. I work really hard. […] I also feel somewhat trapped. I have to keep working at Turk to get the 16 or 17 dollars deposited into my bank account each day. This leaves me no time to find other money earning opportunities. I do not know if using Mechanical Turk for research or other crowd sourcing is ethical or not. I clearly see the dichotomy of not enough pay and yet not wanting this small amount of income to lessen or disappear. Just thinking about it is a source of anxiety.
I would express my desire that the contractors at least remember that those doing work this way are very low paid but still people and to treat them with respect both with regards to writing tone and with regards to paying as high a rate as possible.
Let Them Design Logos
CrowdSpring.com, DesignCrowd.com, are typical examples for the use of contests to organise the workforce. More specifically, both platforms are so called ‘logo mills’, that are aimed at the crowdsourcing of graphic design. There is already more than a dozen of them and 99designs.com is probably the largest. The platform claims to be “the fasted growing design market-place in the world”. It has more than 200,000 registered designers and it already conducted over 180,000 design contests. Even though the site boasts a lot of numbers, the pricing schemes of 99designs are complicated and deliberately opaque in regard to the commission the company takes. It is not directly visible, neither to designers nor clients, that the site actually takes a share of 40 to 45 percent. A client is for example paying $300 for a logo contest and gets on average 116 different finished designs for that money while only one designer gets paid for the work. From the initial $300 99designs takes $120 leaving the designers with a chance of 1 in 116 to eventually getting paid $180. That means the average renumeration comes down to about $1.50 per design, before taxes. There are higher paying contest for things more complex then a logo, but the average money paid out per design on 99designs is $2. Designing a logo usually takes significantly longer then an hour, which means that either, the logos on offer can only be highly derivative, of very low quality or the contributing designers work for even far less then their colleagues toiling away in the micro-payment sweatshops described above. It is, by the way, the external client that decides who wins and if anyone will get paid at all. 99designs offers a 100% money back guarantee if the client doesn’t like the results.
At first sight, there are some similarities with the notorious pitches in architecture. In the documentary Urbanized, Rem Koolhaas bemoans that problem:
“There is an incredible amount of wasted effort in the profession. A fair amount of it is generated through the procedure of competitions which is a complete drain of intelligence. I don’t know of any other profession that would tolerate this. At the same time you are important, we invite your thinking, but we also announce that there is an eighty per cent chance that we will throw away your thinking and make sure that it is completely wasted” (Hustwit, Urbanized min 51:50).
Nevertheless, there are also huge differences between pitches in architecture and crowdsourcing in design. With crowdsourcing in design, it is not just a handful of selected studios competing for one job that will then be paid properly and is prestigious for the studio. Instead hundreds of designers actually complete the job simultaneously and beforehand. But Koolhaas has an important point here: the ethical problem lies not only in the low average wages but especially in the systemic waste of effort and creativity.
There are other crowdsourcing models, also in the design world, in which the contributors become shareholders of the products they help to create (Quirky.com) and others, in which the cash rewards in a contest are significantly higher and the community decides who will get them (Jovoto.com). In other words, there are possibilities to at least mitigate the hardship of crowdsourcing to some extent.
A system such as that of 99designs, however, in which the workers have to gamble for their remuneration, where they have a 1% chance to get paid for their labour while the organisers make a 40% revenue in 100% of the cases, can only be called exploitative and unethical, last but not least because of the way the true price calculation is hidden.There are initiatives such as No!Spec (no-spec.com) that try to prevent designers from participating in so called speculative work, but it is unlikely that these mode of production are going away. There is just too much profit to be made by the platform owners and too much desperation or naivety among those who participate. Even if Christopher Otey should win his case against CrowdFlower, a national class action lawsuit will not be enough against such a global phenomenon, especially if the crowd chooses to be exploited in that way, instead of revolting against it.
Ahn, Luis von et al. “reCAPTCHA: Human-Based Character Recognition via Web Security Measures.” Science 321.5895 (2008): 1465–1468. Web. 20 Jan. 2013.
Airey, David. “Crowdsourced design is a risky business.” Wired UK 2012. Web. 24 Jan. 2013.
Benkler, Yochai. The penguin and the Leviathan : the triumph of cooperation over self-interest. New York: Crown Business, 2011. Print.
Bernays, Edward L, and Mark Crispin Miller. Propaganda. Brooklyn, N.Y.: Ig Pub., 2005. Print.
Bernstein, Michael S. et al. “Soylent: a Word Processor with a Crowd Inside.” Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology. New York, NY, USA: ACM, 2010. 313–322. Web. 20 Jan. 2013. UIST ’10.
Biewald, Lukas. Lukas Biewald of CrowdFlower on TWiST #154 Bonus. 22 June 2011. YouTube Video. http://youtu.be/hhqCVflxkAY
Bogost, Ian. “Gamification Is Bullshit.” Ian Bogost Aug. 2011. Web. 24 Jan. 2013.
—. “Persuasive Games: Exploitationware.” Gamasutra May 2011. Web. 24 Jan. 2013.
Le Bon, Gustave. The crowd : a study of the popular mind. Minneapolis, MI: Filiquarian Publishing, 2005. Print.
Bruns, Axel. Blogs, Wikipedia, Second life, and Beyond : from production to produsage. New York: Peter Lang, 2009. Print.
Carey, John. The Intellectuals and the masses : pride and prejudice among the literary intelligentsia, 1880-1939. London: Faber and Faber, 1992. Print.
Carr, Nicholas. “Sharecropping the Long Tail.” ROUGH TYPE Dec. 2006. Web. 24 Jan. 2013.
Cherry, Miriam. “Working the Crowd.” Concurring Opinions July 2012. Web. 24 Jan. 2013. http://www.concurringopinions.com/archives/2012/07/working-the-crowd.html
Christopher G. Harris. “Dirty Deeds Done Dirt Cheap: A Darker Side to Crowdsourcing.” 2011. Web. 13 Mar. 2012.
Estellés-Arolas, Enrique, and Fernando González-Ladrón-de-Guevara. “Towards an Integrated Crowdsourcing Definition.” Journal of Information Science 38.2 (2012): 189–200. Web. 12 Sept. 2012.
“Experimental Psychology: The Roar of the Crowd.” The Economist 2012. Web. 18 Jan. 2013.
Grossman, Lev. “You — Yes, You — Are TIME’s Person of the Year.” Time 25 Dec. 2006. Web. 15 Jan. 2013.
Herz, J. C. “Harnessing the Hive: How Online Games Drive Networked Innovation.” Esther Dyson’s Release 1.0 18 Oct. 2002 : n. pag. Print.
von Hippel, Eric. Democratizing innovation. Cambridge, Mass.; London: MIT Press ;, 2006. Print.
Howe, Jeff. Crowdsourcing : how the power of the crowd is driving the future of business. London: Random House Business, 2009. Print.
—. “The Rise of Crowdsourcing.” Wired 14 June 2006.
Human Computation. Library of Congress, 2009. Film.
Hustwit, Gary. Urbanized. PlexiFilm, 2011. Film.
Isaac, Mike. “Why Designers Hate Crowdsourcing.” Forbes.com 12 2010. Web. 18 Jan. 2013.
Keen, Andrew. The cult of the amateur : how today’s internet is killing our culture. New York: Doubleday/Currency, 2007. Print.
Kittur, Aniket, Boris Smus, and Robert E. Kraut. CrowdForge: Crowdsourcing Complex Work. 2011. Print.
Kücklich, Julian. “Precarious Playbour: Modders and the Digital Games Industry.” The Fibreculture Journal FCJ-025.5 (2005): n. pag. Web. 14 Jan. 2013.
Lanier, Jaron. “Digital Maoism: The Hazards of the New Online Collectivism.” edge – third culture May 2006.
Leadbeater, Charles. We-think : mass innovation, not mass production. London: Profile Books, 2008. Print.
Leadbeater, Charles, Paul Miller, and England) Demos (Organization: London. The pro-am revolution : how enthusiasts are changing our society and economy. London: Demos, 2004. Print.
Lehrer, Jonah. “Groupthink.” The New Yorker 30 Jan. 2012. Web. 24 July 2012.
Leslie, Ian. “CROWDS R US.” Intelligent Life Dec. 2011. Web. 10 Jan. 2013.
Lessig, Lawrence. Free culture : the nature and future of creativity. New York: Penguin Books, 2005. Print.
—. The future of ideas : the fate of the commons in a connected world. United States: RANDOM HOUSE (MD/TX), 2002. Print.
Lévy, Pierre. Collective intelligence : mankind’s emerging world in cyberspace. Cambridge, Mass.: Perseus Books, 1997. Print.
Mack, Eric. “The Lawsuit That Could Help Undo (or Cement) Crowdsourcing in the U.S.” www.crowdsourcing.org Jan. 2013. Web. 10 Jan. 2013. http://www.crowdsourcing.org/editorial/the-lawsuit-that-could-help-undo-or-cement-crowdsourcing-in-the-us/22968
Mackay, Charles. Extraordinary popular delusions and the madness of crowds. Ware: Wordsworth Reference, 1995. Print.
Massolution. Enterprise Crowdsourcing Research Report. Massolution.com, 2012. Web. 24 Jan. 2013.
McGonigal, Jane. Gaming Can Make a Better World. 2010. presentation. TED Talk.
N. Kaufmann, T. Schulze. “More Than Fun and Money. Worker Motivation in Crowdsourcing–A Study on Mechanical Turk.” (2011): n. pag. Print.
Poetz, Marion Kristin, and Martin Schreier. “The Value of Crowdsourcing: Can Users Really Compete with Professionals in Generating New Product Ideas?” SSRN eLibrary (2009): n. pag. Web. 26 July 2012.
Rand, Ayn, and Nathaniel Branden. The virtue of selfishness : a new concept of egoism. New York: Signet/New American Library, 1970. Print.
Raymond, Eric S. The Cathedral & the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary. 1st ed. Beijing ; Cambridge, Mass: O’Reilly, 1999. Print.
Reicher, Stephen. “The Psychology of Crowd Dynamics.” School of Psychology University of St. Andrews (2001): n. pag. Print.
Reicher, Stephen, and Clifford Stott. “Mad Mobs and Englishmen? Myths and Realities of the 2011 Riots.” the Guardian 18 Nov. 2011. Web. 20 Jan. 2013.
Rheingold, Howard. Smart mobs : the next social revolution. Cambridge, Mass.: Perseus, 2003. Print.
—. The virtual community : finding connection in a computerized world. London: Minerva, 1995. Print.
Schmidt, Florian A. Parallel Realitäten : [ein Einblick in Funktion, Design und Bedeutung computergenerierter Welten]. Sulgen: Niggli [u.a.], 2006. Print.
Schmidt, Florian A, Andreas Töpfer, and Kunsthochschule (Berlin). Kritische Masse von Profis und Amateuren im Design. Berlin: form + zweck, 2010. Print.
Schmidt, Florian, and Danny Pannicke. “Gamer als Designer: Nutzergenerierte Inhalte in Computerspielen.” Informatik-Spektrum 34.6 (2011): 598–606. Print.
Scholz, Trebor. Digital labor : the Internet as playground and factory. New York: Routledge, 2013. Print.
—. “Market Ideology and the Myths of Web 2.0.” First Monday 13.3 (2008): n. pag. Web. 20 Jan. 2013.
Shaugnessy, Haydn. “Work Without Jobs: A Trend For The Educated Elite.” Forbes 2012. Web. 23 Jan. 2013.
Shirky, Clay. Cognitive surplus : how technology makes consumers into collaborators. New York, NY [u.a.]: Penguin Books, 2011. Print.
—. Here comes everybody : how change happens when people come together. London: Penguin, 2009. Print.
—. “How cognitive surplus will change the world.” TED. 2010. Video on TED.com.
—. “Shirky: A Group Is Its Own Worst Enemy.” O’Reilly Emerging Technology. 2003.
Sloane, Paul. A guide to open innovation and crowdsourcing : expert tips and advice. London; Philadelphia: Kogan Page, 2011. Print.
Sotamaa, Olli. “On Modder Labour, Commodification of Play, and Mod Competitions.” First Monday 12.9 (2007): n. pag. Web. 20 Jan. 2013.
Steiner, Christopher. “The Creativity of Crowds.” Forbes.com 16 Feb. 2009. Web. 18 Jan. 2013.
Surowiecki, James. The wisdom of crowds : why the many are smarter than the few. London: Abacus, 2005. Print.
Tapscott, Don, and Anthony D Williams. Wikinomics : how mass collaboration changes everything. London: Atlantic, 2008. Print.
Terranova, Tiziana. “Free Labor: Producing Culture for the Digital Economy.” Electronic Book Review (2003): n. pag. Web. 17 Jan. 2013.
Toffler, Alvin. The third wave. New York: Bantam Books, 1989. Print.
Turner, Fred. From counterculture to cyberculture : Stewart Brand, the Whole Earth Network, and the rise of digital utopianism. Chicago, Ill.; Bristol: University of Chicago Press ; University Presses Marketing [distributor], 2008. Print.
Unterberg, Bastian. “Crowdsourcing (Jeff Howe).” Social-Media-Handbuch Theorien, Methoden, Modelle und Praxis. Baden-Baden: Nomos, 2010. 121–135.
Wales, Jimmy. As Wikipedia moves to S.F., founder discusses planned changes. Nov. 2007. San Francisco Chronicle. http://www.sfgate.com/business/article/As-Wikipedia-moves-to-S-F-founder-discusses-3233536.php#ixzz2IaSu23G2