Mechanical Turk had a good run, but not surprised it's shutting down. I'm sure the platform was getting flooded with people doing task arbitrage and using lots of AI anyway.
I believe the issue is that this can no longer be a horizontal play. MTurk was mostly for unskilled tasks...the kind AI can do well enough that it isn't worth the cost differential to verify it or keep farmed to humans. The "trust but verify" AI output is now the kind that requires domain expertise. This is what most full stack AI companies are bringing to industries.
Curious if this kind of work will come around again one day or was just a moment in time. If it does I'm sure it will be specifically about generating training data.
I'm unsure. If it does, it will be work that is too expensive or inaccurate or regulatory for current AI methods. For example, you want a doctor to sign off on some AI output on a diagnosis.
However, I'm not sure a single platform will be how it emerges
As AMT's largest requester for the past 10 years, this news was relayed to requesters at the same time as respondents. It's also worth noting that our lead contact, the Sr Program Manager at AWS leading AMT, transitioned to Amazon Bedrock and SageMaker Model Evaluations a ~2-3 years ago.. Leaving behind essential zero team managing the project after they migrated over the stored value accounts to native AWS billing.
Initially the annotation of biomedical literature (https://pubmed.ncbi.nlm.nih.gov/25592589/) but left academia for commercial use where we transitioned to arbitrage the market research and ephemeral task markets. It was essentially a research panel with a underdeveloped UI for tasks.. so abstract away AMT's tooling so it can be used by any buyer in the ResTech, Political Polling and Data Annotation space
My favorite part of AMT is always going to be figuring out that if we only paid in $0.07 intervals, their commission algorithm would round down to the nearest whole cent when their 20% commission resulted in a fractional cent on the unit transaction level, not the monthly invoice level. Was ultimately worth it to have implemented https://git.generalresearch.com/panels/amt-jb/tree/jb/flow/a...
It’s kind of crazy they’re shutting this down just when this Service probably has the most possibilities ever. You have an agent with Multiple people doing actual physical tasks in the real world seems like something that could be really powerful.
I believe the issue is that this can no longer be a horizontal play. MTurk was mostly for unskilled tasks...the kind AI can do well enough that it isn't worth the cost differential to verify it or keep farmed to humans. The "trust but verify" AI output is now the kind that requires domain expertise. This is what most full stack AI companies are bringing to industries.
Curious if this kind of work will come around again one day or was just a moment in time. If it does I'm sure it will be specifically about generating training data.
"This robot is having trouble folding a tshirt help it out for 1$"
Unless they go the waymo route of highly trusted people but I think mass deployed robots are a bit safer than a car for this.
However, I'm not sure a single platform will be how it emerges
My favorite part of AMT is always going to be figuring out that if we only paid in $0.07 intervals, their commission algorithm would round down to the nearest whole cent when their 20% commission resulted in a fractional cent on the unit transaction level, not the monthly invoice level. Was ultimately worth it to have implemented https://git.generalresearch.com/panels/amt-jb/tree/jb/flow/a...
Mercor has a market value of $20B basically doing the same thing but desperately trying to find workers.
They are terribly monotonous tasks.
I can see why it's shutting down if its still the same thing.
Lllms could probably do everything there without rotting out minds for basically pennies.