AI: cautionary tales

A few weeks ago, I wrote about the tentative first steps in our AI journey.

One of the things we’re trying to do is to automate and shorten some of our workflows, so we created prompts (using Claude) that would take an article prepared in Word and convert it to html, automatically applying styling, paragraph and header tags.

Early results were encouraging.

But then a couple of recent hiccups made me realise that we needed to proceed with great caution.

The first was when Claude added a hyperlink which had not been in the original text. It was in effect creating content, when all we wanted it to do was process the content we’d served it with.

As it happens, the link worked and made sense, but what if it hadn’t?

Realising that it might add content, not just process it, was shock # 1.

Shock # 2 came when we asked Claude to process a document, and it ended up leaving bits out!

Specifically, it was the ‘A:’ at the start of each of the answers in this week’s Q&A article.

Aaaaaaaaaaagh!!!!!!!

What was going on? Was Claude trying to be helpful? Or clever? Was the prompt poorly phrased?

Whatever it was, the realisation that Claude, and presumably other AI tools, could add or take away content, when you wanted neither of those things to happen, was an eye-opener.

My takeaways:

• Oversight and quality control is critically important. Don’t assume that your prompt is watertight. It might be one day, but it’s unlikely to be yet. Everything needs to be carefully checked.
• Keep refining your prompts. When problems are found, the prompt needs to be revisited. Don’t start relying on workarounds.
• Follow best practice when designing your prompts. ‘Prompt engineering’ is a rapidly evolving discipline but there’s lots of advice online, some of it good. A prompt needs to be carefully crafted, tightly phrased and logically laid out. If your prompt is the proverbial dog’s dinner, then perpetual frustration awaits...