Welcome to Part I of Edition No. 68 of my weekly newsletter, providing practical analysis in the world of digital content strategy.
Analysis: Getting To Know Meta’s AI Assistant
Meta’s stock recorded the largest ever single-day gain in market value earlier this month after its quarterly earnings call.
One way in which it’s impressing is with its AI investments. Today, I’m going to dig into its AI assistant, Meta AI, and compare it to ChatGPT.
Originally announced in September, Meta AI was touted as “An Assistant That Spans Our Apps and Devices.”
"In text-based chats, Meta AI has access to real-time information through our search partnership with Bing and offers a tool for image generation”
In retrospect of this experiment, ChatGPT 3.5 (the free version) would have been a fairer fight, but I did in fact use ChatGPT 4 (the paid version) for all you see below.
My Approach: Ask Meta AI how it could help me, take it up on those offers, then go through the same process with ChatGPT 4.
Summary Conclusion: Meta AI is a fun and oftentimes useful companion to (in this case) Instagram. If you’re looking for heavy-duty automation assistance, though, ChatGPT (or Gemini) remains the better option.
But if you need some quick answers or to brainstorm new ideas, Meta AI is just fine.
Whether it’s worth going multiple layers into the Instagram app just to interact with the bot is worth it is another question. (And the answer is, not really.)
Here are the details of the experiment:
I began by asking Meta AI what it could do to help me. I wanted to give it a chance to set the parameters of our test and set it up for success.
I then took it up on each of the eight offerings.
I’m going to rate Meta on each of these tasks from 1 to 5, 5 being the best.
Here’s a summary – with more details below – based on my limited sample size.
Task 1: History/Questions: 4
Task 2: Writing Drafts: 3
Task 3: Translations: 2
Task 4: Suggestions/Brainstorming: 4
Task 5: Grammar and Vocabulary: 3
Task 6: Definitions and Explanations: 5
Task 7: Everyday tasks: 2
Task 8: Chatting: 3
For context, I performed this exercise on the desktop browser version of Instagram. It’s much easier to type as well as copy and paste text that way vs. on a phone.
1 Answering questions on various topics like science, history, and more
Rating: 4/5
Better than ChatGPT? In using citations, yes. In depth of info, no.
I began by asking about a piece of history that is known, but not common knowledge. To keep the bot honest, I also asked about something on which I’m fairly familiar.
Based on this one-question sample size, the information is accurate and well-presented.
What I really love, though, is the ability to dig in further (See more on Bing)…
…and see the sources. (The links are clickable.)
I’m also impressed by the way it consolidated and rephrased my question instead of plopping it verbatim into Bing.
Going directly to a search engine (or even ChatGPT) is still more convenient than opening Instagram > opening my DMs > finding the Meta AI bot > entering my query. Even so, if you’re already in the app, this is a great experience.
ChatGPT, on the other hand, blew MetaAI away in both it’s depth of information as well as the way it was laid out in different sections with multiple bullet points.
On the other hand, I had to ask for links to sources, which it did not provide by default. That’s still an improvement from older versions of ChatGPT.
This will become a theme, but for quick, digestible answers, Meta was the clear winner. If you want to dig deeper, though, ChatGPT is the way to go.
2 Generating text, like articles, emails, or even creative writing
Rating: 3/5
Better than ChatGPT? No
Asking the bot to help me write an article seemed like the obvious choice, but I wanted to go with a more original challenge: composing an email to request quotes for tree-stump removal (something I did recently).
I wish I would have done this before sending my emails because it came up with things I didn’t think to ask.
It also wrote in a much more natural and human way than I’m used to with ChatGPT.
While I did later ask for help with an article, I got a half-baked, one-paragraph summary that missed a lot of the points I asked it to cover. Multiple revision requests yielded the same meager results.
It wasn’t capable of improving an existing article, either.
To rewrite small, specific sections of text in a more human way, I would prefer Meta AI. But to save time and create something that resembles real content (while still needing human editing), ChatGPT is far superior.
3 Translating text from one language to another
Rating: 2/5
Better than ChatGPT? Not really
I became fluent in Spanish and conversational in Catalan after my five years in Barcelona. This, then, seemed like a great opportunity to put the bot to the test.
I didn’t want to try basic phrases like “where is the bathroom?” but rather, idioms used in these languages that wouldn’t come through clearly with a literal translation.
Spanish phrase: Me cago en la leche
Literally: I poop in the milk
Meaning: This is what you say as a reaction to something bad happening. It means something along the lines of, “That really messes things up.” Or “We’re screwed.”
It’s overall explanation is good, but the translation is not accurate. This is used as a reaction to a bad situation, not to say you don’t care about something. (“No me importa” or “me da igual” would be better for that.
I also don’t love that it doesn’t show sources. If you relied on Meta to understand this phrase, you would use it incorrectly.
•
Spanish phrase: Ostras
Literally: Oysters
Meaning: Shoot! Or Holy cow! It can be expressed as regret or surprise to something.
Much better than the last one, though this is a single word. (It’s funny that it used shucks – which has a double meaning – to describe “oysters,” though it’s correct in doing so.)
It’s not mildly offensive, but could be considered unprofessional.
•
Catalan Phrase: He begut oli
Literally: I drank oil
Meaning: I messed up.
While there are places that tell you this means what Meta says it means, that’s not my understanding. (I can’t go as far as to say that no one uses the phrase in that way.)
More typically, though – as the English-speaking news source in Catalunya points out – this means that you have failed at something.
ChatGPT gave similar answers to all three inquiries, but its explanations were better. I wouldn’t, however, rely on either for any non-straightforward translations.
4 Offering suggestions and ideas for brainstorming
Rating: 4/5
Better than ChatGPT? No
Rather than dive into all five strategies, I asked for help expanding on the first suggestion.
Both the websites the bot cited as well as a simple Google search shows that Bing does in fact value social media interactions related to a particular piece of content, while Google maintains that it does not.
•
I didn’t have to go too far down the rabbit hole to get useful, practical tips. Even then, the more I dig, the more specific the bot gets. While I wouldn’t rely on Meta AI to run your SEO strategy, it’s a good place to start.
ChatGPT, however, can do this more efficiently and with better content.
5 Helping with language-related tasks like grammar and vocabulary
Rating: 3/5
Better than ChatGPT4? No
I planted multiple misspellings or odd phrasings in the first few paragraphs of this article from the Tennessean.
While it didn’t catch all of them, it got most. It also made some other more subjective but solid suggestions.
I wouldn’t rely on Meta AI to be my final proofreader, but it would certainly save me some time.
When given the same task, ChatGPT4 was gave more details, and also caught the it’s/its error, which Meta did not.
6 Providing definitions and explanations
Rating: 5/5
Better than ChatGPT4? Tie
I was impressed by both LLMs on this task. They both guessed the exact word I was thinking of with limited information, then followed up with strong definitions and (closely matching) synonym lists.
7 Assisting with everyday tasks, like setting reminders or conversions
Rating: 2/5
Better than ChatGPT: No
This was the most disappointing/underwhelming work done by Meta AI. Let’s just say I’m glad my life didn’t depend on getting a reminder to make coffee!
This is what I consider to be the equivalent of the conversation:
Bot: “Would you like $1,000?”
Me: “Why yes, I would love $1,000.”
Bot: “Great! Here, have $1,000.”
Me: “…OK. I didn’t receive any money.”
Bot: “I apologize for the mistake! If you want $1,000 you’ll have to drive to the bank and withdraw it from your account using the ATM. Let me know if you need directions to your bank.”
ChatGPT was more forthcoming and self-aware. It also better grasped the task of reminding me to make coffee, not simply telling me the coffee awaits.
8 Chatting and having a friendly conversation!
Rating: 3/5
Better than ChatGPT? It depends.
In this use case, Meta AI is like an attractive person that’s fun to date, but not someone with whom you would want to spend the rest of your life.
ChatGPT, on the other hand, has more substance. It enhances the conversation. Maybe it’s not as funny and lacks common sense, but it brings home the bacon and gets the job done.
I also asked Meta AI what it meant by “conversions” – it’s No. 7 offering – and it showed some sense of humor without being prompted.
Keep reading with a 7-day free trial
Subscribe to Gerick News(letter) to keep reading this post and get 7 days of free access to the full post archives.