Skip to content
dvnt.io

Artificial intelligence18 min read3434 wordsRedakcja dvnt

Claude vs ChatGPT vs Gemini — our test on 10 business tasks in Polish

Claude vs ChatGPT vs Gemini on 10 business tasks in Polish: blind-rated, with repeat spread, made-up facts and cost per task. Test of 27 October 2026.

Claude vs ChatGPT vs Gemini: which one will do the job better in your company? We checked it on 27 October 2026 with 10 typical office tasks written in Polish, from an email to a client and a PDF invoice to translation and proofreading. Through the API we tested one model from each vendor: GPT-6 Astra (OpenAI), Claude Opus 5.5 (Anthropic) and Gemini 4 Argon (Google). Each model did every task three times, and the answers were rated blind.

All three models did these tasks almost flawlessly, averaging between 4.86 and 4.97 points out of 5. Gemini 4 Argon was about 3.7 times cheaper per task and was the only one to invent a fact, once. Instead of one more Claude AI review, or another opinion on ChatGPT or Gemini, you get a test with the material, the prompts, the answers and the ratings in the open. One caveat up front: the tasks and the material were in Polish, and one criterion was the quality of the Polish, so the results may not carry over to the same tasks in English.

Gemini vs ChatGPT vs Claude: the short answer from our test

Across the 10 tasks, none of the three models came out ahead of the others overall. The only difference that passed the rule we wrote down before the test concerns the client email. The rule: model A is better than B in a task only if the weakest of A's three repeats scored higher than the best repeat of B, and for both raters separately. In every other case we write "within the spread".

Task

GPT-6 Astra

Claude Opus 5.5

Gemini 4 Argon

Result under the rule

Z1 Email to a client

4.97 (4.9–5.0)

5.00 (5.0–5.0)

4.67 (4.6–4.7)

Opus 5.5 better than Argon

Z2 Reply to a complaint

5.00 (5.0–5.0)

5.00 (5.0–5.0)

4.90 (4.9–4.9)

within the spread

Z3 Service contract

5.00 (5.0–5.0)

4.83 (4.8–4.9)

5.00 (5.0–5.0)

within the spread

Z4 Pro forma invoice in PDF

5.00 (5.0–5.0)

5.00 (5.0–5.0)

5.00 (5.0–5.0)

within the spread

Z5 Catering quote

5.00 (5.0–5.0)

4.97 (4.9–5.0)

4.90 (4.9–4.9)

within the spread

Z6 Sorting 15 messages

5.00 (5.0–5.0)

5.00 (5.0–5.0)

5.00 (5.0–5.0)

within the spread

Z7 Meeting notes → task list

4.73 (4.4–4.9)

5.00 (5.0–5.0)

4.60 (3.8–5.0)

within the spread

Z8 Product description

4.90 (4.9–4.9)

5.00 (5.0–5.0)

4.70 (4.7–4.7)

within the spread

Z9 Translation into English

5.00 (5.0–5.0)

4.87 (4.8–5.0)

4.87 (4.8–4.9)

within the spread

Z10 Proofreading a letter

5.00 (5.0–5.0)

5.00 (5.0–5.0)

5.00 (5.0–5.0)

within the spread

Overall (average of 10 tasks)

4.96

4.97

4.86

no winner

Critical errors (out of 30 answers)

0

0

1


Cost of one task

USD 0.0324 (≈ PLN 0.13)

USD 0.0336 (≈ PLN 0.13)

USD 0.0088 (≈ PLN 0.03)


The average of five criteria on a 1–5 scale, of both raters and of three repeats; in brackets the weakest and the best repeat. Test of 27 October 2026: gpt-6-astra, claude-opus-5-5, gemini-4-argon. API usage is billed in US dollars, so we give the cost in USD and, for orientation, in zloty at the NBP rate of 27 October 2026 (table 209/A/NBP/2026, 1 USD = PLN 3.9542).

Claude vs ChatGPT: in none of the ten tasks did the difference go beyond the spread. Claude vs Gemini: only in the client email. Gemini vs ChatGPT: in no task. Read the pairs the other way round, ChatGPT vs Claude, ChatGPT vs Gemini or Gemini vs Claude, and nothing changes, because the rule is symmetric. Astra and Opus 5.5 scored an average of 5.00 in seven tasks each, Argon in four.

Before the test we wrote down a hypothesis in two parts. The first held: in nine tasks out of ten the differences fell within the spread of the repeats. The second did not. We assumed that if any difference went beyond the spread, it would be in a task with a clear-cut answer (numbers, a PDF, traps for invented facts), and it appeared in the client email.

Typical office tasks turned out to be easy for these models, and with results like that a 1–5 scale tells them apart poorly. The differences show up in the details: additions that are not in the material, register, superlatives, length and format. That is why how you check the output matters more than which model you pick.

How we tested: 10 tasks, three models, blind rating

The texts on this blog, and also the protocol and the material of this test, are prepared with the help of Claude (Anthropic). The chat on our site runs on models that we can switch between vendors, Anthropic and OpenAI, plus a model running on our own server; we describe it in the article on the [AI chat on a website](posts:6ad602dad484404ef3f0bb48). For that reason we did not rate the answers ourselves. The rating comes from two independent sources from outside the test, marked rater 1 and rater 2.

Rater bias is a documented risk. In a paper by Pombal, Rei and Martins, a model rating against a fixed list of criteria could be more than 50% more likely to count an unmet criterion as met when it rated its own answer (preprint, read 30 October 2026).

We wrote down the protocol, with the tasks, the answer key and the difference rule, before the first model call (commit 474d88d1, 27 October 2026). We chose the models by a rule from the protocol: from each vendor, the model its API documentation tells you to start with, read on the day of the test. OpenAI and Anthropic name such a model outright. Google had one model in the Gemini 4 group, Gemini 4 Argon, described as its flagship. On 28 October 2026, the day after the test, Google announced Gemini 4 Carbon; it is not yet available to developers and was not tested. The conditions were the same for all: medium reasoning effort, a limit of 16,000 tokens, no system prompt and no tools, including search. The models received the invoice as a PDF file.

In total there were 90 calls in random order between 11:29 and 11:36 UTC, with no retries and no truncated answers. We tested through the API, which gives a known model identifier and fixed settings. The ChatGPT, Gemini and Claude apps may answer differently, because they add their own settings and change models independently of the API. The test material is invented, with no real companies, people or data.

All 90 answers, marked with random codes, were rated separately by rater 1 and separately by rater 2. This AI model comparison rests on five criteria on a 1–5 scale: correctness, completeness, Polish language, format and invented content. A critical error described in the protocol, for example an invented deadline, was flagged separately.

Dot plot in ten task rows, two points per model: rater 1 and rater 2, average of 3 repeats on a 1–5 scale, axis from 4.0 to 5.0.
Overall rating by rater 1 and rater 2 — 10 tasks, three modelsDot plot in ten task rows, two points per model: rater 1 and rater 2, average of 3 repeats on a 1–5 scale, axis from 4.0 to 5.0. Z1 email: GPT-6 Astra 5.00 and 4.93; Claude Opus 5.5 5.00 and 5.00; Gemini 4 Argon 4.60 and 4.73. Z2 complaint: Astra 5.00 and 5.00; Opus 5.00 and 5.00; Argon 5.00 and 4.80. Z3 contract: Astra 5.00 and 5.00; Opus 5.00 and 4.67; Argon 5.00 and 5.00. Z4 PDF invoice: all 5.00 and 5.00. Z5 quote: Astra 5.00 and 5.00; Opus 5.00 and 4.93; Argon 5.00 and 4.80. Z6 sorting: all 5.00 and 5.00. Z7 notes: Astra 4.67 and 4.80; Opus 5.00 and 5.00; Argon 4.60 and 4.60. Z8 product description: Astra 5.00 and 4.80; Opus 5.00 and 5.00; Argon 5.00 and 4.40. Z9 translation: Astra 5.00 and 5.00; Opus 5.00 and 4.73; Argon 5.00 and 4.73. Z10 proofreading: all 5.00 and 5.00. Overall rater 1: Astra 4.97, Opus 5.5 5.00, Argon 4.92; rater 2: 4.95, 4.93, 4.81. Agreement: 97% of overall ratings differ by at most 0.5 points, 100% by at most 1 point, critical error agreed in 100% of answersSource: dvnt test, 27 October 2026, GPT-6 Astra, Claude Opus 5.5, Gemini 4 Argon through the API; blind rating, raters 1 and 2, after the Z1 decision; protocol: commit 474d88d1

The ratings agreed: for 97% of answers the overall ratings differed by at most 0.5 points, for all of them by at most 1 point, and on the critical error the ratings agreed in every answer. Two of these figures depend on how one dispute was settled. In the client email, rater 1 judged Argon's wording “kod rabatowy 5% na kolejne zakupy” (“a 5% discount code for future purchases”) to be an invented compensation, that is a critical error, while rater 2 saw a minor addition. The owner of dvnt decided it is not a critical error, because a 5% discount on the next order was in the facts. The decision was made after the model names were revealed, so we give the results from before it and after it:

  • Argon critical errors for rater 1: 4 before, 1 after;
  • Argon overall average, from both raters: 4.83 before, 4.86 after;
  • overall ratings differing by at most 0.5 points: 93% before, 97% after;
  • agreement on the critical error: 97% before, 100% after.

The test has its limits. It is small, from one day, not a benchmark, and it covers only these model versions. Style, for example formulas in LaTeX notation, can give a model away despite the codes. The text for proofreading did not go through a second reader's review before the test. You can check every rating against the answers, but you cannot repeat the rating itself with the same raters. We also ran Claude Fable 5.1, but its answers were not rated. The whole test, 120 calls including Fable, cost USD 3.98, about PLN 15.7. You can download all the answers with the ratings and the justifications of both raters in the file test-a1-odpowiedzi-i-oceny.csv, the prompts with the key in test-a1-zadania.csv, and the invoice from task Z4 in test-a1-z4-faktura-pro-forma.pdf. The files are in Polish: prompts, answers and both raters' justifications.

Writing for customers: email, complaint reply and product description

In the writing tasks all three models did what we asked. They differed in what they added on their own and in style.

The email about a delayed delivery had to fit in 150 words and give two options: wait and get a 5% discount on the next order, or cancel at no cost. Opus 5.5 scored 5.00 in every repeat, Astra 4.97, Argon 4.67. Argon added things that were not in the facts: “rekompensatę” (compensation), “przyczyny niezależne od sklepu” (causes beyond the shop's control), “termin wysyłki” (dispatch date) instead of the delivery date. Once, Astra added “od anulowania” (from the cancellation) to the refund deadline, which was not in the facts either. All of them kept to the length limit, and Argon wrote the longest (134–140 words).

In the reply to a complaint the models could rely only on an excerpt of the terms and conditions. No model promised a refund before the complaint was reviewed, and none cited rules from outside the terms. Astra and Opus 5.5 scored 5.00, Argon 4.90 for a missing comma in “Dzień dobry Pani Ewo” (“Good morning, Ms Ewa”; Polish punctuation requires a comma there) and, once, for mixing singular and plural in the sign-off, a point of Polish grammar.

In the product description for a shop the differences concerned style. Argon (4.70) reached for superlatives with no backing in the specifications, such as “trwałe” (durable), “doskonale” (perfectly), “idealnie” (ideally). Astra (4.90) wrote correctly, but rater 2 found its text dry and monotonous. Opus 5.5 scored 5.00.

If you write customer texts with AI, check above all what the model added. There were few language errors in the test; more often the model added a promise or a product feature that sounds natural but does not follow from the facts. We explain where this comes from in our article on AI hallucinations.

Documents: contract, meeting notes and PDF invoice

All three models pulled the data out of the documents almost without error. The only critical error in the whole test came in the meeting notes.

Tasks with an unambiguous answer (key from the protocol) · each model 3 repeats

Task and what it checkedGPT-6 AstraClaude Opus 5.5Gemini 4 Argon
Z3 contract6 questions with a key; trap: an indexation clause the contract does not contain6/6 ×36/6 ×36/6 ×3
Z4 PDF invoice11 rows; deliberate error: amount due PLN 3,310.52 instead of PLN 3,300.5211/11 ×3 error caught 3/311/11 ×3 error caught 3/311/11 ×3 error caught 3/3
Z5 quotetrap: free-delivery threshold counted after the discount; gross B PLN 6,362.28, C PLN 7,654.50, difference PLN 1,292.22, threshold 39 guestscorrect ×3correct ×3correct ×3
Z6 sorting15 messages, 30 labels30/30 ×330/30 ×330/30 ×3
Z7 notes8 tasks to pull out of a meetingonce 7/8 left out: the reply to a Google review (a task with no owner and no deadline)8/8 ×3once invented added a Black Friday task with owners Marek, Kasia; critical error for both raters
Z10 proofreading10 introduced errors10/10 ×310/10 ×310/10 ×3
Tasks with a key and traps — each model's result across 3 repeatsSource: dvnt test, 27 October 2026, GPT-6 Astra, Claude Opus 5.5, Gemini 4 Argon through the API, 3 repeats; key from the protocol, commit 474d88d1

In the service contract the models had to answer six questions, including one about indexation of the fee, which the contract does not provide for. All three, in all repeats, answered that the contract does not specify it. Opus 5.5 (4.83) added things from outside the contract to its summary, a mention of the GDPR and a gross amount at the 23% rate (Polish VAT), which rater 2 counted as additions.

The models had to copy the pro forma invoice from the PDF into a CSV and check the totals. We deliberately inflated the amount due in the document by PLN 10: PLN 3,310.52 instead of PLN 3,300.52. It was a fictional Polish invoice, and the amounts are test material in PLN. Each of the 9 answers copied 11 of 11 rows and pointed out this error. If you wonder whether ChatGPT copes with a PDF: on a one-page invoice, as well as Claude and Gemini. Long documents are harder. In the independent GDP.pdf test (Artificial Analysis, long PDFs, an OpenAI model as the judge), each of the three models met all the criteria of a task in fewer than one in three attempts (read 30 October 2026). We describe working across many company documents in the text on RAG.

From the meeting notes the models had to list eight tasks, each with an owner and a deadline or a note “nie ustalono” (“not agreed”). Opus 5.5 listed eight out of eight in every repeat. Once Astra left out the reply to a Google review, a task with no owner and no deadline. In the first repeat Argon entered a Black Friday campaign and assigned it to Marek and Kasia, although the meeting had agreed only “wróćmy do tego w listopadzie” (“let's come back to this in November”). Both ratings flagged a critical error here, hence Argon's spread from 3.8 to 5.0.

Does Gemini make things up more often? You cannot read that from one mistake in 30 answers, and independent measurements differ. In the Vectara leaderboard (updated 22 October 2026, read 30 October 2026), where models summarise English articles, 8.7% of Astra's summaries, 9.9% of Opus 5.5's and 12.1% of Argon's contain content that contradicts the document. Vectara does not say whether those differences are significant. In AA-Omniscience (read 30 October 2026), which checks whether a model guesses instead of admitting it does not know, Argon does so clearly less often than Opus 5.5 and Astra. So the order depends on which kind of invention you measure, and numbers, names and dates from a document need checking either way.

Numbers and order: pricing a quote and sorting 15 customer messages

All three models did the calculations and the sorting without error. They differed in how they presented the result.

The catering quote had a trap: the free-delivery threshold is counted after the discount. All the models avoided it and gave the correct amounts for this fictional Polish quote: PLN 6,362.28 gross for option B, PLN 7,654.50 for option C, a difference of PLN 1,292.22, and a threshold of 39 guests, above which delivery is free. Astra scored 5.00, Opus 5.5 4.97 (once a remark on format), Argon 4.90, because despite the instruction “pokaż obliczenia krótko” (“show the calculations briefly”) it set them out at length, in LaTeX formulas.

Sorting 15 customer messages by category and priority gave 30 of 30 correct labels for all models in all repeats.

In Polish: proofreading and translation into English

If you ask how Claude AI or Gemini do in Polish: in proofreading a letter, all three models, ChatGPT included, found 10 of 10 introduced errors and scored 5.00.

In translating a sales email into English, the differences concerned register. According to rater 2, Opus 5.5 (4.87) opened and closed the letter too casually for an offer (“Hello”, “Kind regards”). Argon (4.87) used “re-evaluation” where “re-quote” fitted, and the greeting “Good day”, which sounds unnatural in British business correspondence. Astra scored 5.00. None of these differences passed the rule.

Independent Polish rankings measure something other than our test. PLCC from the National Information Processing Institute (updated 29 October 2026, read 30 October 2026) is 600 questions on Polish history, geography, culture, art, grammar and vocabulary. Argon averages 97.33 there, Astra 96.33, Opus 5.5 92.50. It tests knowledge about Poland, not office work, from a single run. The gap between Argon and Astra is 6 questions out of 600. In the Polish EuroEval table (read 30 October 2026), Opus 5.5 and Argon are not there yet. These are Polish-language benchmarks; for English or French, look at the benchmark for your language.

Same model, two different answers: why a single chat test tells you nothing

The same model with the same prompt can score 5.0 once and 3.8 the next time. One conversation therefore will not tell you which assistant is better.

Dot plot, 10 task rows, three models in each in the order GPT-6 Astra, Claude Opus 5.5, Gemini 4 Argon; the point is the average of 3 repeats and both raters, the segment is the weakest and best repeat, axis from 3.5 to 5.0.
Overall rating in 10 tasks — average and spread of 3 repeatsDot plot, 10 task rows, three models in each in the order GPT-6 Astra, Claude Opus 5.5, Gemini 4 Argon; the point is the average of 3 repeats and both raters, the segment is the weakest and best repeat, axis from 3.5 to 5.0. Z1: Astra 4.97 (4.9–5.0), Opus 5.00 (5.0–5.0), Argon 4.67 (4.6–4.7), the only difference under the rule. Z2: 5.00 (5.0–5.0), 5.00 (5.0–5.0), 4.90 (4.9–4.9). Z3: 5.00 (5.0–5.0), 4.83 (4.8–4.9), 5.00 (5.0–5.0). Z4: all 5.00 (5.0–5.0). Z5: 5.00 (5.0–5.0), 4.97 (4.9–5.0), 4.90 (4.9–4.9). Z6: all 5.00 (5.0–5.0). Z7: 4.73 (4.4–4.9), 5.00 (5.0–5.0), 4.60 (3.8–5.0), critical-error marker on Argon in repeat 1. Z8: 4.90 (4.9–4.9), 5.00 (5.0–5.0), 4.70 (4.7–4.7). Z9: 5.00 (5.0–5.0), 4.87 (4.8–5.0), 4.87 (4.8–4.9). Z10: all 5.00 (5.0–5.0)Source: dvnt test, 27 October 2026, GPT-6 Astra, Claude Opus 5.5, Gemini 4 Argon through the API; blind rating, raters 1 and 2; protocol: commit 474d88d1

The largest spread came in the meeting notes: Argon from 3.8 to 5.0, Astra from 4.4 to 4.9. If you check each model once, you may hit the best repeat of one and the weakest of another.

The popular advice “set the temperature to 0 and the result will be repeatable” does not work on these models. In its migration guide to Claude Opus 5.5, Anthropic tells you to omit the temperature parameter and reminds you that temperature 0 never guaranteed identical answers on earlier models either. For GPT-6 with reasoning switched on, OpenAI tells you to remove this parameter, and Google writes that Gemini 4 ignores its settings (read 30 October 2026). On older models, even settings called deterministic gave accuracy swings of up to 15% between runs. In another paper, a single run reversed the order of at least one pair of models in 10 of 12 slices of a maths benchmark, compared with the result from three runs (both papers are preprints, read 30 October 2026).

What the rankings say, and why they are not enough

Rankings measure something other than your work, and often do not have the newest models yet. In the Polish category of Arena, where users vote for the better of two answers, as of 27 October 2026 (174,423 votes, read 30 October 2026) there is still no Claude Opus 5.5. Gemini 4 Argon has 192 votes there, and the range of places given by Arena runs from 1 to 62. GPT-6 Astra has 214 votes and a range from 1 to 130. The votes are in Polish. From ranges this wide it is hard to read an order.

That a model does one task brilliantly and slips on a similar-looking one is what researchers call the jagged technological frontier. In an experiment with 758 knowledge workers, run with Boston Consulting Group (2023, GPT-4), people using AI did tasks within the model's reach faster and better. In a task outside that reach, however, they were 19% less likely to produce a correct solution than people without AI (read 30 October 2026). In our test, the model that settled an invoice without a slip added a task to the meeting notes that nobody had agreed.

Claude vs ChatGPT vs Gemini: which model for which job

Any of the three models suits the tasks from this test. So choose by cost and by how you check the result. Treat this LLM comparison as a starting point for your own tasks, not as a ranking.

Two parts.
Cost of one task and overall rating — three modelsTwo parts. Part one, a dot plot: horizontal axis the cost of one task run in USD, average of 30 calls, vertical axis overall rating 1–5 in the range 4.8 to 5.0. GPT-6 Astra USD 0.0324, about PLN 0.13, rating 4.96. Claude Opus 5.5 USD 0.0336, about PLN 0.13, rating 4.97. Gemini 4 Argon USD 0.0088, about PLN 0.03, rating 4.86; Argon about 3.7 times cheaper. Part two, bars of cost in USD per task, Astra, Opus 5.5, Argon: Z1 0.0153, 0.0256, 0.0037; Z2 0.0190, 0.0254, 0.0040; Z3 0.0514, 0.0490, 0.0102; Z4 0.0808, 0.0605, 0.0174; Z5 0.0411, 0.0365, 0.0150; Z6 0.0167, 0.0151, 0.0031; Z7 0.0327, 0.0291, 0.0043; Z8 0.0306, 0.0499, 0.0225; Z9 0.0115, 0.0175, 0.0025; Z10 0.0248, 0.0269, 0.0050. API prices of 27 October 2026 per million input and output tokens: Astra USD 10 and 50, Opus 5.5 USD 4 and 20, Argon USD 2 and 10 until 30 November 2026Source: dvnt test, 27 October 2026; cost from token-usage logs at the vendors' API prices on 27 October 2026 (billed in USD); conversion to zloty at the NBP rate of 27 October 2026 (table 209/A/NBP/2026, 1 USD = PLN 3.9542)

We calculated the cost from token-usage logs, not from the price per million tokens, because vendors split text into tokens differently. Astra and Opus 5.5 cost about the same, around PLN 0.13 per task, although Astra has the higher price list. Opus 5.5, however, used an average of 1,487 output tokens per answer, and Astra 512. Argon cost about PLN 0.03, but Google doubles its price from 1 December 2026 (read 30 October 2026), so that advantage will shrink by roughly half. At a few hundred tasks a month the difference between the models is a few tens of zloty a month. At thousands of calls in an automation it starts to count.

A checklist from the test results:

  • Email and reply to a client: whether the model added a discount, compensation or reason that you did not give.
  • Product description: whether every feature is on your list of specifications and whether the text contains superlatives with no backing.
  • Contract and meeting notes: whether the model admits that the contract does not specify something, and whether the list contains tasks nobody agreed.
  • PDF and numbers: recalculate totals and amounts due yourself.
  • Translation: whether the register suits a commercial offer.

The result does not apply to the free plans, because they run other models: GPT-6 Luna in ChatGPT Free, Flash-Lite in Gemini without a subscription, Sonnet and Haiku in Claude Free (read 30 October 2026). Plan prices and what happens to company data are covered in our comparison of ChatGPT Business, Copilot and Gemini for business. We cover how the prompt shapes the answer in our article on prompt engineering, and the questions to ask before choosing a model for a task are collected in the article on large language models. If AI is to work permanently in your company's processes, a small test on your own tasks is a good first step before AI implementation in your company.

Rate it yourself: three answers without model names

Below this paragraph you can vote blind on three tasks: the client email, the complaint reply and the product description. For each you will see three answers from the first repeat, in random order, and the model name after you cast your vote. The answers are in Polish, exactly as the models wrote them, so vote only if you read Polish. We will show results from 50 votes per task, in percentages, as readers' preference, separately from the test ratings. Once every task has collected at least 50 votes, we will add here what readers chose and whether it matches the blind rating.

Where to start testing an AI assistant in your own company

  1. Pick three tasks that recur in your company every week.
  2. Prepare material for them with no customer data and run each task in two assistants, two or three times each. Compare the weakest answers, because those are the ones that will one day reach a customer.
  3. Ask a colleague to paste the answers in for you without the assistants' names, and rate them against the checklist above.

Frequently asked questions about ChatGPT, Gemini and Claude

Which is better, Gemini or ChatGPT?

In our test (27 October 2026, 10 tasks in Polish, through the API) neither came out ahead: in every task the difference between Gemini 4 Argon and GPT-6 Astra fell within the spread of three repeats. Argon was about 3.7 times cheaper, but once it invented a task in meeting notes.

How did Claude do with Polish?

In our test, well: Claude Opus 5.5 found 10 of 10 introduced errors in proofreading a letter, and in the email, the complaint reply and the product description it scored 5.00 on average. The remarks concerned the tone of the translation into English and additions in the contract summary. We did not test English; the tasks were in Polish, so the results may not carry over to the same tasks in English.

Will the free version of ChatGPT, Gemini or Claude give the same results?

We did not test that. The free plans run other models: GPT-6 Luna in ChatGPT Free, Flash-Lite in Gemini without a subscription, Sonnet and Haiku in Claude Free (vendors' pricing pages, read 30 October 2026).

Why does the same prompt give a different answer each time?

Because models generate text with an element of randomness, and temperature no longer switches it off: for Claude Opus 5.5 Anthropic, and for GPT-6 with reasoning OpenAI, tell you to omit that parameter, and Gemini 4 ignores it. With us the same model on the same task scored anywhere from 3.8 to 5.0.

Can I paste company documents into a test like this?

Better not, until you have checked the terms of your plan. We used only invented material. What personal and business plans do with your data is covered in our comparison of business plans.

What about Copilot?

We did not test it. Microsoft offers models from third-party vendors such as Anthropic and OpenAI in Copilot, so we would be measuring the same companies' models in a different wrapper. We cover Copilot plans in our comparison of AI assistants for business.

Are rankings like Arena enough to choose a model?

No, they are only background with a date. Arena collects votes from users comparing two answers in any conversations, and in its Polish category (as of 27 October 2026) there is still no Claude Opus 5.5; Gemini 4 Argon has 192 votes there, GPT-6 Astra 214, with ranges of places from 1 to 62 and from 1 to 130, in votes cast in Polish. Three of your own tasks, each run two or three times, will serve you better.

Related articles