Skip to main content

We should all be worried about AI infiltrating crowdsourced work

A new paper from researchers at Swiss university EPFL suggests that between 33% and 46% of distributed crowd workers on Amazon’s Mechanical Turk service appear to have “cheated” when performing a particular task assigned to them, as they used tools such as ChatGPT to do some of the work. If that practice is widespread, it may turn out to be a pretty serious issue.

Amazon’s Mechanical Turk has long been a refuge for frustrated developers who want to get work done by humans. In a nutshell, it’s an application programming interface (API) that feeds tasks to humans, who do them and then return the results. These tasks are usually the kind that you wish computers would be better at. Per Amazon, an example of such tasks would be: “Drawing bounding boxes to build high-quality datasets for computer vision models, where the task might be too ambiguous for a purely mechanical solution and too vast for even a large team of human experts.”

Data scientists treat datasets differently according to their origin — if they’re generated by people or a large language model (LLM). However, the problem here with Mechanical Turk is worse than it sounds: AI is now available cheaply enough that product managers who choose to use Mechanical Turk over a machine-generated solution are relying on humans being better at something than robots. Poisoning that well of data could have serious repercussions.

“Distinguishing LLMs from human-generated text is difficult for both machine learning models and humans alike,” the researchers said. The researchers therefore created a methodology for figuring out whether text-based content was created by a human or a machine.

The test involved asking crowdsourced workers to condense research abstracts from the New England Journal of Medicine into 100-word summaries. It is worth noting that this is precisely the kind of task that generative AI technologies such as ChatGPT are good at.

We should all be worried about AI infiltrating crowdsourced work by Haje Jan Kamps originally published on TechCrunch



source https://techcrunch.com/2023/06/18/i-robot/

Comments

Post a Comment

Popular posts from this blog

The Silent Revolution of On-Device AI: Why the Cloud Is No Longer King

Introduction For years, artificial intelligence has meant one thing: the cloud. Whether you’re asking ChatGPT a question, editing a photo with AI tools, or getting recommendations on Netflix — those decisions happen on distant servers, not your device. But that’s changing. Thanks to major advances in silicon, model compression, and memory architecture, AI is quietly migrating from giant data centres to the palm of your hand. Your phone, your laptop, your smartwatch — all are becoming AI engines in their own right. It’s a shift that redefines not just how AI works, but who controls it, how private it is, and what it can do for you. This article explores the rise of on-device AI — how it works, why it matters, and why the cloud’s days as the centre of the AI universe might be numbered. What Is On-Device AI? On-device AI refers to machine learning models that run locally on your smartphone, tablet, laptop, or edge device — without needing constant access to the cloud. In practi...
1 comment
Read more

Apple’s AI Push: Everything We Know About Apple Intelligence So Far

Apple’s WWDC 2025 confirmed what many suspected: Apple is finally making a serious leap into artificial intelligence. Dubbed “Apple Intelligence,” the suite of AI-powered tools, enhancements, and integrations marks the company’s biggest software evolution in a decade. But unlike competitors racing to plug AI into everything, Apple is taking a slower, more deliberate approach — one rooted in privacy, on-device processing, and ecosystem synergy. If you’re wondering what Apple Intelligence actually is, how it works, and what it means for your iPhone, iPad, or Mac, you’re in the right place. This article breaks it all down.   What Is Apple Intelligence? Let’s get the terminology clear first. Apple Intelligence isn’t a product — it’s a platform. It’s not just a chatbot. It’s a system-wide integration of generative AI, machine learning, and personal context awareness, embedded across Apple’s OS platforms. Think of it as a foundational AI layer stitched into iOS 18, iPadOS 18, and m...
Post a Comment
Read more

Max Q: Anomalous

Hello and welcome back to Max Q! Last week wasn’t the most successful for spaceflight missions. We’ll get into that a bit more below. In this issue: First up, a botched launch from Virgin Orbit… …followed by one from ABL Space Systems News from Rocket Lab, World View and more Virgin Orbit’s botched launch highlights shaky financial future After Virgin Orbit’s launch failure last Monday, during which the mission experienced an  “anomaly” that prevented the rocket from reaching orbit, I went back over the company’s financials — and things aren’t looking good. For Virgin Orbit, this year has likely been completely turned on its head. The company was aiming for three launches this year, but everything will remain grounded until the cause of the anomaly has been identified and resolved. It’s unclear how long that will take, but likely at least three months. Add this delay to Virgin’s dwindling cash reserves and you have a foundation that’s suddenly much shakier than before. ...
Post a Comment
Read more