fbpx

AI Chat Model Comparison

I independently asked Chat GPT, Microsoft Copilot and Google Gemini each to do a comparative feature and sentiment analysis between these models and Anthropic's Claude. I then fed all three responses to Claude and asked it to synthesize them into a simple comparison giving consideration to the usefulness to the average small business. I then used Claude and then Chat GPT Plus to turn that into a quiz in HTML. The results are in this post.

Share:

data codes through eyeglasses

The below is provided purely for experimental demonstration purposes. Do not trust it to be accurate or use it to make significant decisions.

I independently asked Chat GPT, Microsoft Copilot and Google Gemini each to do a comparative feature and sentiment analysis between these models and Anthropic’s Claude. I then fed all three responses to Claude and asked it to synthesize them into a simple comparison giving consideration to the usefulness to the average small business. I then used Claude and Chat GPT Plus to turn that into a quiz in HTML. The results are below.

Claude generated quiz: Provided as a single piece of code. Gives too many “no good recommendation” responses. I ran out of tokens to refine it until later… I’ve now updated it and the updated version follows after this one:

AI Model Selection Quiz

AI Model Selection Quiz

This quiz will help you determine the best AI model for your business needs.

Claude generated quiz: Second iteration:

Here is the updated version after I prompted Claude this way: "This code gives too many “no good recommendation” responses. Update the logic and subsequent code so that it always picks the best option given the parameters chosen AND never gives a “no good recommendation” response." Notably, it doesn't work. This is a good demonstration of why this type of functionality is helpful up to a point, but if I do not know how to debug the code myself I need further help to get this functional. Because I am using the free version of Claude, I also ran out of my message length so I was unable to ask Claude to help me debug it either. Chat GPT comes out ahead on this one, in part because I am paying for pro. Results on that are further down this page.

AI Model Selection Quiz

AI Model Selection Quiz

This quiz will help you determine the best AI model for your business needs.

Claude also gave me this response along with the above code:

"Here's how the updated code works:

  1. Instead of using a single recommendation object with keys based on the exact response combination, the code now uses an array of recommendation objects. Each object contains the model name, a list of criteria, and the corresponding recommendation text.
  2. When the user completes the quiz, the checkAndShowResult function first tries to find an exact match by checking if all the criteria in any of the recommendation objects are present in the user's responses. If a match is found, it displays the corresponding recommendation.
  3. If no exact match is found, the code looks for a fallback match based on the primary goal (the first user response). It checks if any recommendation object has the primary goal as one of its criteria and displays the corresponding recommendation.
  4. If no fallback match is found, a generic recommendation is displayed, suggesting exploring a combination of AI models and recommending personalized guidance from your team.

With this updated logic, the quiz will always provide a suitable recommendation based on the user's responses, prioritizing the most important criteria (primary goal, cost-effectiveness, ethical considerations, etc.). It will never show the "no good recommendation" response.

Note: You may need to adjust the criteria and recommendation texts in the recommendations array based on your specific requirements."

Chat GPT generated quiz: (It gave me three separate code blocks for HTML, CSS, and Javascript. So I asked it for a single block of code with CSS and Java embedded.) It also asked about Microsoft integration for Copilot. But did not ask about Google Workspace integration from Gemini. So I asked it to fix that and to also double-check that it was not self-biasing to give more weight to suggesting Chat GPT than the provided comparison data warranted.

AI Model Recommendation Quiz for Small Businesses

What is your primary use case for AI?

Do you require integration with Microsoft 365 apps?

Do you require integration with Google Workspace?

Do you need support for multimodal content (text, images, audio, video)?

Since Claude gave a detailed description of how it intended to rewrite the code. I asked Chat GPT to rewrite its code using the same logical description Claude gave me. I was curious to see if it would generate a working code since Claude did not. I noticed right away there was no closing. It got closer, but clicking the Get Result button doesn't do anything. Below is the result:

Somehow one of these codes has also broken the footer on this page of my website. Another demonstration of how these tools can sometimes create new problems as we try to solve others.

AI Model Recommendation Quiz for Small Businesses

What is your primary use case for AI?

Do you require integration with Microsoft 365 apps?

Do you require integration with Google Workspace?

Do you need support for multimodal content (text, images, audio, video)?

📍Proudly located in Waynesville, NC 28786. Serving Western North Carolina including Asheville and beyond to Nationwide.

Hornsby Creative is located on un-ceded ancestral lands of the Eastern Band of the Cherokee Indians, S’atsoyaha and Miccosukee. We acknowledge that this tribal nation and people continue to thrive in this area.