best 8 data labeling and rlhf vendors

snorkel flow's premise is programmatic supervision: you encode labelling rules and weak signals rather than paying people to annotate every example, which cuts manual volume substantially. that is a genuinely different answer to a category where the ethical and cost problems both scale with human hours.

it also sells expert data-as-a-service, contracting phds and subject-matter specialists directly rather than routing through a crowd platform, positioned for rlhf ranking, code evaluation and creative judgement. we found no litigation or regulatory action against snorkel, which distinguishes it from four vendors here — though absence of found reports is weaker evidence than a clean finding.

the gaps are the category norm. no pricing at all, no worker locations, no wage floor, and no data-ownership terms we could reach. worth knowing that its $100m series d in may 2025 at a $1.3bn valuation included in-q-tel, the cia's venture arm, among the investors — not a criticism, but a fact some buyers weigh.

argilla is annotation software rather than an annotation service. hugging face acquired it in june 2024 for a reported $10m, and the core remains free and open source, deployable on your own hugging face space or infrastructure, tightly integrated with the hub.

that architecture answers the data question cleanly — datasets stay under your control, and there is no vendor in the path reusing them — and it removes the pay-transparency problem by removing the workforce. no contractors, no wage floor to publish, no disputes to report.

it also cannot do the job the other seven do. somebody still has to label the data, and argilla supplies the interface rather than the hands. if you don't have in-house annotators you will end up hiring a vendor from elsewhere on this page anyway, and inheriting that vendor's labour practices along with its output. rlhf is a supported workflow; the human effort behind it is yours to source.

labelbox pairs its annotation platform with alignerr, its expert contractor marketplace, claiming 250,000-plus specialised hours a month spanning entry-level work up to medical and legal domain experts. a third-party review site reports contractor pay of roughly $15 to $60 an hour depending on domain — the highest reported band among the crowd-model vendors here, though labelbox publishes no figure itself.

the same review site reports unpaid or low-paid multi-hour evaluation stages before contributors reach paid projects. we found no court filing or regulatory action on this, so it stands as aggregated worker anecdote rather than a documented dispute, and we're labelling it that way rather than either amplifying or ignoring it.

it raised a $110m series e at roughly a $1bn valuation in october 2025 led by softbank vision fund 2 and a16z, and grew from 232 to 551 employees between 2024 and mid-2026. its pricing page returns 404, and its data-processing terms weren't reachable either, so who owns the labelled output is unverified.

toloka runs an open crowd platform across a claimed 100-plus countries and 40-plus languages, where workers sign up directly and complete microtasks. for sheer breadth and unit economics at volume, nothing else here matches it, and it reportedly serves amazon, microsoft and anthropic.

worker pay is where the model shows. third-party review sites report roughly $1 to $6 an hour for casual tasks, rarely above $8 even for experienced workers, with typical monthly earnings from $20 to $400 depending on intensity. toloka publishes no wage floor and no country-level pay breakdown, so those figures are worker-reported rather than disclosed. we found no litigation specific to toloka.

the ownership is easy to miss. toloka was spun out of yandex and became part of nasdaq-listed nebius group; a $72m investment led by bezos expeditions in may 2025 ended nebius's majority voting control while nebius retained a significant majority economic stake. for buyers with jurisdictional sensitivities, that history is worth knowing before it comes up in a security review.

surge built to a reported $1.2bn revenue run-rate without raising venture capital, which in a category defined by capital intensity is unusual and gives its founder full control. it focuses on rlhf, rl environments and language annotation for frontier labs, reportedly including openai, anthropic, meta and microsoft, working through around 50,000 expert contractors alongside roughly 130 employees.

contractor pay is reported by third parties at 30 to 40 cents per working minute — roughly $18 to $24 an hour if the work is continuous — which is well above crowd rates and below mercor's published expert bands. surge publishes no wage floor and no country breakdown.

a class action alleges surge misclassified data annotators as independent contractors, denying benefits, with claims of unpaid training time and time limits tight enough to reduce effective pay. the case is pending and no liability has been determined. worth noting the shape of the allegation matters more than the vendor: the same claim is live against mercor and was made against scale.

invisible blends human workers with automation across several products, with meridial described as a network of 24,000-plus vetted experts and a dedicated ai training and rlhf service line. it raised $100m in september 2025 at a reported valuation above $2bn, and cites soc 2, hipaa and gdpr compliance.

worker reviews on indeed and glassdoor describe being treated as employees while classified as contractors, delayed payments, and time-tracking software not disclosed until after a contract is signed. these are aggregated anecdotes rather than court filings — we found no litigation or regulatory finding — and we report them as what they are, while noting the pattern matches allegations that have been formally pleaded against other vendors here.

one practical warning for anyone researching this: there is a name collision. a separate, smaller company also called invisible, working on ai-agent infrastructure, was acquired by perplexity in august 2025. several aggregators conflate the two, which would tell you this vendor had been acquired when it had not.

mercor is the only vendor here that publishes rates at all: bands of roughly $60 to $120 an hour for specialised roles, with reporting citing over $2m paid daily across its network and 60 to 70% of top-line revenue passed to contractors. after seven vendors publishing nothing, that deserves credit, and it is why the entry isn't last.

the security record is the reason it is seventh. a 2026 supply-chain compromise of litellm, attributed to the lapsus$ group, exfiltrated roughly four terabytes of data reportedly including social security numbers, dates of birth, passports, biometric face and voice data and banking details. at least six to seven class actions followed, including gill v. mercor in the northern district of california filed 1 april 2026. a separate suit, cox v. mercor io corporation filed october 2025, alleges undisclosed monitoring software installed on contractors' personal computers without reimbursement.

a proposed federal class action in texas additionally alleges independent-contractor misclassification on behalf of around 30,000 experts including physicians, attorneys, bankers and engineers. all of these are pending and no liability has been determined. mercor was reported in july 2026 to be raising at a $20bn valuation.

on capability scale is the leader: the deepest capacity for frontier training data and rlhf, the most capital, and the most government contract wins including a department of defense engagement reported above $300m. a ranking built on throughput would put it first, and this one is not.

the ownership changed the market. meta took a reported 49% stake for about $14.3bn in june 2025, valuing scale at $29bn, and founder alexandr wang left to lead meta's superintelligence lab. google, openai and microsoft were reported to have reduced or ended engagements afterwards over confidentiality and competitive concerns. scale still operates separately, but a large minority owner is now a direct rival to several of its customers.

the labour record is the longest here. remotasks operations in the philippines have been documented by the business and human rights resource centre with pay as low as around a cent for short tasks and reports of withheld payments; the kenya operation shut abruptly in march 2024 with wages reportedly owed and the country's data labelers association weighing action. two us suits were filed in january 2025 — one alleging wage violations and misclassification with effective pay near $15 an hour, one over the mental-health toll of content-moderation work at outlier. all pending, no liability determined. scale cut around 200 employees and 500 contractors in july 2025 while pivoting toward defence.