Job Security

This morning, on our last day of school, I was talking to my fellow sixth-grade English teachers. "You know what's going to be a game-changer for you?" I asked, pointing at our youngest member. "Soon there is going to be AI grading!"

Her eyes widened and the three of us considered, for a moment, what it would be like to have a machine do the hardest, most time-consuming, part of our job. Not only would it save us countless hours, but we could use the instantaneous feedback to coach students to make the changes and corrections in real-time.

"Right now you can assess writing using the standard rubrics they have on the site, but in a few years?" I continued, "You'll probably be able to put in your own rubric and have the writing graded instantly!"

I was thinking about our conversation and my promise to explore the options when I sat down with my laptop this afternoon. No time like the present, I thought and typed in a few search terms. I was rewarded with free trials of two AI grading sites, one of which claimed to have custom rubrics, but which really only allowed me to choose the categories I wanted assessed from a standard list. With that one, I was also able to select the level of work I was submitting, elementary, middle, or high school, and the genre.

I started by copying and pasting a few of my students' writing. The results, for middle school, were in the C range, even though I had given it a B+. When I shifted it to elementary, the results were closer. The feedback was detailed and mostly accurate, but not to the point where I would recommend using it to either my colleagues or students.

Then I decided to run my own writing through, and I pasted a few blog entries. I was dismayed to receive Cs and Ds on my work! 

Let me give you an example. Here is a piece of writing from March 2009, back when I started writing my daily slice of life.

It's rainy and cold here. Last weekend we had unseasonably springy temps in the 70's, but when I woke up this morning, this gray Saturday reminded me a little bit of those endless weekends when I was a kid. My gosh, back then it seemed like the raw, wet, late-winter weather went on forever, and come March, my brother and sister and I ran out of things to do in the house by early Saturday afternoon. My parents had plenty to keep them busy, however, and so I have memories of a lot of wandering through the house, sighing. I probably threw myself on the couch or my bed a few times, too.

I have no children of my own, but I am aunt, by blood and friendship, to more than a dozen kids, aged 1 to 16. They are fun to have around any time-- an afternoon, overnight, weekend, or even for a week or three in the summer. In fact, we have a couple of god-daughters here for the weekend right now.

We had a great day today. We made waffles for breakfast, caught a super-corny magician/comedian at the local cinema and drafthouse, hiked a national park with the dog, watched a couple of movies, and then spent some more kitchen time on homemade pasta, meatballs and sauce. Up next? Roasting marshmallows in the fireplace for s'mores-- cause that's what aunties are for.

Here's what app.essaygrader.ai had to say when I ran it through at high school level:

Evaluation:

Organization: The essay lacks a clear organizational structure. The ideas are presented in a random and haphazard manner, making it difficult for the reader to follow the flow of the essay. There is no clear introduction or conclusion, and the paragraphs do not transition smoothly from one to another.

Progression: The essay lacks a clear progression of ideas. The writer jumps from one topic to another without providing any logical connection between them. The lack of progression makes it difficult for the reader to understand the main point or purpose of the essay.

Development of Ideas: The essay lacks sufficient development of ideas. The writer provides vague and general statements without providing any specific details or examples to support them. The essay would benefit from more specific and concrete examples to illustrate the writer's points.

Conventions: The essay contains several grammatical errors and lacks proper punctuation. There are also inconsistencies in verb tense usage. The writer should proofread the essay carefully to correct these errors.

Language and Vocabulary: The language and vocabulary used in the essay are informal and colloquial. The writer should strive for a more formal and academic tone in their writing. Additionally, the essay lacks variety in sentence structure and word choice, making it repetitive and monotonous.

Suggestions:

- Organize the essay into clear paragraphs with topic sentences and supporting details. This will help the reader follow the flow of the essay more easily.

- Develop the ideas more fully by providing specific examples and details. This will make the essay more engaging and persuasive.

- Proofread the essay carefully to correct grammatical errors and improve punctuation.

- Use more formal and academic language and vocabulary in the essay. This will make the writing more professional and credible.

- Vary sentence structure and word choice to make the essay more interesting and engaging to read.

Estimated Grades:

Organization: D

Progression: D

Development of Ideas: C

Conventions: D

Language and Vocabulary: C

Overall Grade: D

I guess I get that a machine can't appreciate the impressionist style of this slice of life, but wow! The sentence structure and word choice comment stings. As for the several grammatical errors,  I had to run a separate report to see what they were. 

Error: temps - Suggestion: temperatures Error: this gray Saturday - Suggestion: this gray Saturday, Error: a little bit - Suggestion: a little Error: those endless weekends - Suggestion: these endless weekends Error: come March - Suggestion: when March came Error: brother and sister - Suggestion: brothers and sisters Error: in the house - Suggestion: around the house Error: early Saturday afternoon - Suggestion: early on Saturday afternoon Error: however, - Suggestion: However, Error: so I have memories - Suggestion: so I have many memories Error: of a lot of wandering - Suggestion: of wandering Error: through the house - Suggestion: through the house, Error: probably threw - Suggestion: probably threw myself Error: or my bed - Suggestion: or on my bed Error: aged 1 to 16 - Suggestion: aged one to sixteen Error: any time-- - Suggestion: anytime-- Error: cinema and drafthouse - Suggestion: cinema and draft house Error: a national park - Suggestion: the national park Error: Up next? - Suggestion: What's next? Error: cause - Suggestion: because Error: aunties - Suggestion: aunts

As you can see, many of the "errors" are misreads by the AI, and would be perfectly understandable to a human reader. Some of them actually contribute to the style of the piece. 

Ultimately, I wouldn't use any of the feedback to revise this piece, because it's clear that the app didn't understand the intention and standards of the genre. But in the not too distant future? I bet it will.