Skip to main content

Humanify

☰
How Does AI Detection Work for Essays

All AI Plagiarism detection tools use the same two metrics to identify text generated by AI. The latter is the predictability of the text word by word. The second is variation of sentences throughout the entire work. If the reading is low on both, the writing was likely written by a computer—AI writing typically delivers low scores on both measures.

Tools vary in the amount of weight, or score, which is given to each signal, hence the same essay can earn a 10 on one tool and a 90 on another. These are the only two ways in which AI checkers and AI detection software for essays work. There are some detectors with a third layer, and a few AI vendors have tried a fourth. The following blog will discuss about how AI detection works for essays and how to avoid it.

Different Methods to Evaluate AI Content

1. Perplexity

Perplexity: a measure of surprise. A language model will look at your sentence and predict what the next word in your sentence should be. When it guesses correctly nearly every time, perplexity is low.

By design, a large language model (LLM) like ChatGPT will have low perplexity, because it is trained to select the next word it thinks is most likely. Human writing is muddled. We choose odd words, interrupt ourselves and insert a reference not expected by a model.

Two sentences that convey the same meaning.

  • AI: Technology enhances the education system by making it more efficient and accessible.
  • Human: Tech really accelerates learning. Also, it helps you to avoid thinking.

There is nothing wrong with the first sentence. The problem is that it is just the same way a model would generate it.

2. Burstiness 

Burstiness indicates the variation in sentence length and structure throughout a text. Human typers type in spurts. Long sentence with three clauses, followed by a short sentence, followed by a sentence fragment.

Weather conditions are more reliable. They are all of the same length, same shape and the rhythm stays the same from the first paragraph to the last. Sentence variation is more important than any word choice because that evenness is a signal to detectors.

3. Classifiers that have been trained against a baseline written by a human

No single signal is used for either of these signals. Both are given as input to a machine learning model (a classifier) which has been trained using large collections of labelled text, in this case human essays, serving as the baseline text, and output from the model as the labelled text.

The classifier is trained on statistical characteristics such as the density of transitions, paragraph structure, etc., and on return it gives a probability. The essence of how the AI writing detectors works is simple and that’s what it’s good at and that’s where it fails. They gave their answers to unedited, default output of a model they were trained on. They are significantly weaker when dealing with modified text (edited, rewritten, newer versions than the sets used to train them).

4. Watermarking, and why it rarely matters

There are other ways, such as watermarking. The provider doesn’t rely on patterns; it creates patterns based on its own word selection, but they are not visible to the reader, but noticeable by the provider’s tools. Google DeepMind’s SynthID is currently in production for at least some of the Gemini output, while OpenAI has stated it could do it but didn’t because it believed it would have a negative impact on non-native speakers.

Watermarks will not be there if the provider does not add them, and they will not work if the text is rewritten. The vast majority of the essays tagged today were tagged on patterns, and not on a watermark.

How to tell if an essay is AI-generated?

Essays are flagged more readily than most writing, and not due to students’ cheating practices. Academic writing has an almost dehumanizing tone, follows predictable transitions, has an outline that looks like a five-part essay, and has little personal voice, just like the thing detectors are programmed to find. This is how AI essay detectors differ from a generic web detector. They are not tuned for essays, but they do find more of what they’re looking for there.

The qualities that attract the attention of a flag are:

  • A tone effect which remains constant from beginning to end.
  • At a density that human writers do not so often encounter, transitions such as furthermore and in addition.
  • A uniform number of words in each paragraph of the text.
  • No specific example, opinion or detail that only you would have.

Section type is also a factor, and formulaic passages suffer the most. Literature review sections were shown to be particularly susceptible to false positives in academic research contexts as a formal summary of others’ work is statistically similar to a model output. Length is also important. Turnitin increased the size of the minimum from 150 to 300 words, because the more words there are, the more accurate the results will be, and reported false positives in the first and last few sentences of a document.

How to avoid getting an AI-generated essay caught in plagiarism detection?

The guaranteed solution is not a ruse. It is trying to reproduce the last text, which is what the detector is measuring. Go to the manual pass, which is free and effective.

Edit it first, then share it!

AI can be used for the outline, for the direction of your research or for a paragraph that you are struggling on. Restate the body in your own words.

Include something that only you can include – example from your course, an opinion, a number from your own reading.

Break the rhythm. Place a 5-word sentence beside a 25-word sentence by design.

Reduce the stock conversions. Usually, the words “Furthermore and in addition” can simply be dropped. 

FAQs

Can AI plagiarism checkers identify plagiarized paraphrased sentences?

Partly. Use of paraphrasing tools increases perplexity, thereby decreasing scores on the weaker detectors. That is the pattern that stronger classifiers are studying – sentence structure generally remains the same during paraphrasing.

Which AI score do I need to be concerned with?

No, not the number, it depends on the school. Turnitin considers less than 20 percent originality as too unreliable to display and most education providers require teachers to start a discussion instead of taking any action based on the score. A high reading is not a condemnation, it’s a sign to review your draft and maintain your version history.

Can a detector detect my essay if I have used AI for only the outline?

Usually not. The words in front of the detector are scored and there is no trace of planning in the finished sentences. Outline, reading list, or prompt asking for where your argument is weak will not be apparent in a scan.

Which AI essay checker do I rely on for my essay?

The best one will depend on the tool used, as every tool has its own threshold. The tool you will need to use will be the one your school uses – and for the majority of pupils the tool will be Turnitin which you cannot use.

Does it matter if the score changes when the essay is rewritten?

It doesn’t change the structure unless it changes it. Switching synonyms doesn’t really change the reading, as the sentence forms remain unchanged. Reconstructing sentences get it a lot farther. On a passage of fewer than a few hundred words, however, you can forget about any score you earn later as all detectors will be unreliable on such a short text.

How to verify the essay AI score before submission?

Copy and paste in a free detector or upload the file. Most tools will produce an AI detection essay report highlighting the sentences that they detected, which is more helpful than the percentage alone. It’s easy to cross check more than one, because scores vary, and if they’re able to cross check multiple, you don’t have to do that by hand.

In Conclusion

The art of writing is what AI writers excel at.AI writing detection is all about statistics. A detector calculates the predictability of your words and the variability of your sentences, then compares the two, and gives back a probability. It is right so often that it can be used, and it is wrong so often that formal, careful and non-native writing gets caught in it.

The same habits are helpful to you in both directions: write the substance yourself, save your first draft, change sentence structure, and proofread before anyone else does. When a draft isn’t sounding good and the deadline is near, look at the score before running it through a humanizer on a small section of it and comparing.